Customizing Grid Header

Is there a way we can add an icon to the Grid Header title (similar to the sort icon)?

We have criteria that is set via a form for the grid. Some of these fields are also in the grid.

We'd like to indicate via the grid column header if this field currently has criteria set or not - something like a filter icon.

How can we hook into and append something to the grid header title the way that the sort icon seems to do?