Do not use any detergents as they would create a microfilm on the filter fibres. You also do not want to breath the detergent either..
Instead a pure solvent like denaturated alcohol should do. It should not damage the fibers or any glue fixing the filter in place if not left for too long. But still dissolve the microfilm of the aromate stuff that stuck to it.
Use a pan or something, put it into if for a few minutes, shake a bit, then partial-dry with a hair dryer on lowest (non-heating) setting. Repeat once more and full-dry in the vac.
Do not put the wet filter in the vac. First pre-dry it by a dryer and then put in vac to full-dry by 5 minutes on no load. That will dry any left-over places and protect the glue if it started being affected.
If alcohol is not enough, go with Acetone. But careful with time there - it is way more aggresive.
The problem is some woods contain lots or organic aromate stuff which is oily and sticks to everything. The only way to get rid of it is to dissolve it in some solvent.
EDIT:
Above assumes you got the white plastic fibres filter. If you have a paper main filter - then tough luck.