WebGPU filter
given a pretend table of cells, with columns for position, subclass and a pretend “gene_x”, and a table of connections between those cells (given by indexes start and end), with an associated strength, the following demo applies a simple filter via a WebGPU compute shader. the shader is generated using a short, SQL-like builder-style api:
const { table, clause, select, all } = given({ cells: { subclass: 'u32', gene_x: 'f32', position: 'vec2f' }, edges: { start: 'u32', end: 'u32', str: 'f32' },}).from('edges');const filter = select('$index') .select(table('cells').at('start').dot('gene_x')) .select(table('cells').at('start').dot('subclass')) .select(table('cells').at('end').dot('subclass')) .where( all(clause(table('cells').at('end').dot('subclass'), '==', 'toClass')) .and(clause(table('cells').at('start').dot('subclass'), '==', 'fromClass')) .and(clause(table('cells').at('start').dot('position'), 'all(>=)', 'minCorner')) .and(clause(table('cells').at('start').dot('position'), 'all(<)', 'maxCorner')) ) .build(device);As you can see, for an edge to pass the predicate and be included in the results, its starting cell must be within the box from minCorner to maxCorner, and the subclass of the cell at the start and end of the edge must match fromClass and toClass respectively. Note also the multiple select statements - the requested fields will be packed into the result buffer in the requested order, and displayed in the table after you press the run! button.
why is this cool?
This is cool for a few reasons! 1. the above SQL-query is mostly typesafe, and it expands to quite a lot more (boilerplate) code that we dont want to write over and over again. 2. WebGPU lets us apply the predicates in parallel - we can run these queries in a browser, without blocking the main thread, with a substantial performance benefit (assuming there is enough data to actually justify the GPU overhead) vs. a javascript for-loop. 3. This process converts column-major data into row-major data - which might be helpful for data-download purposes. 4. We can use this to drive more sophisticated rendering pipelines - by doing a preliminary filter via a compute shader, we can connect the results to a rendering pass (the data never has to leave the GPU).the filter system can also be used for basic aggregation. The following will create a 2D table of values - each row/column value will represent a distinct subclass value. The subclass of the cells at the start of all edges make up the columns, and the subclass of the cells at the end of all edges become the rows.
const { table, groupBy, column } = given({ cells: { subclass: 'u32', gene_x: 'f32', position: 'vec2f' }, edges: { start: 'u32', end: 'u32', str: 'f32' },}).from('edges');// instead of using select, use groupBy on up to two u32 expressions:const aggregate = groupBy(table('cells').at('start').dot('subclass'), table('cells').at('end').dot('subclass')) .sum(column('str'), '$count', '$unused', '$unused') .build(device);note that although min, max and sum are supported, you cannot mix aggregation operations (this is a hardware
imposition) - in the above example, we can take the sum of up to 4 values concurrently. The resulting aggregation tool
can be applied to the results of a previous filtering result, by selecting the $index and passing it to the
aggregator. this is not done in the example to illustrate that aggregating the example in this case (edges) is about 10x
slower than filtering!
Caveats
Aggregation is done via a render pass, rather than a compute shader, for annoying reasons that we wont get into here. That means that the format of the results is constrained by renderable, blendable, webGPU texture formats. note the ‘$unused’ tags - these map to the RGBA position within a “pixel” of the resulting histogram. in the example, because$unused is set for the B and A components, the underlying system will choose a 2-component texture - in this case
rg32float.
*
$count cant be used in min/max, because it would always be 0. * $count will be interpreted as f32 if the
other values selected for aggregation are f32, otherwise it will be interpreted as u32. * prefer using $unused in
the later channels - choosing something like sum($unused, $unused, $unused, something) will force the system to pick a
4-component texture, when a single-channel texture would have worked. * the expected type of the G,B,A components is
based on the inferred type of the R component - setting it as count or unused will likely result in a type error. * the
aggregation system requires float32-blendable (if you attempt to aggregate floating point data), so the build will fail
if that feature is not supported by the given GPUDevice.