The original poster got lucky and m1 and m2 can in fact be constructed.

But your overall post and skepticism is warranted. It's definitely wrong in the general case the original post made.

To solve this easily, you truly need the 128-byte table from vpermi2v / AVX512 (note that ASCII is a 7-bit code and thus all functions of ASCII fit inside of a 128-table).

The reason the original post got lucky is that the top 2 bits of ASCII determine special vs numbers vs lowercase vs uppercase.

The bottom 5 bits determine the specific character. ASCII is a table of tables.

0x30-0x39 are '0' - '9' respectively. So 0x3 in the 'top' table proves you have a number.

0x40 is uppercase and 0x60 is lower case letters.

Bottom table is harder to construct but we don't actually need to know the top bits. We just need 0x0-0x9 (valid for numbers) vs 0x1-0xF (valid for 0x4 and 0x6, as 0x40 and 0x60 are invalid letters), and finally 0x0-0x1A (aka p-z, valid for top half of alphabets in ASCII).

This harder bottom table is solvable with 3 bits of the 8, meaning we have 5 spare bits for future extensions.

So valid for numbers is bit 0x1. Valid for lower-half alphabet is 0x2 and valid for upper-half alphabet is 0x4.

Locations like 5 are valid for all three (0x7) while locations like 0 are only valid for numbers and upper half alphabet (aka 0x5).

----------

In the general case, the [0-9a-zA-Z] language is a regular language (even in an arbitrary non-ASCII code in binary), which means it can always be solved with a state transition table and a finite automata. And yes, pshufb can be a very useful tool for making such a table (and a bigger table like vpermi2v is even better).

But it's not clear that the pshufb sequence in the original post would always work for an arbitrary jump table. Indeed, for best results you likely need to use pext and other complex bit instructions to efficiently extract the most info out of the bits available.

Or you know.... Use the 128-tanle from vpermi2v.

------

Note: FPGAs are composed of 4-LUTs which share many similarities with these 4-bit / 16-byte tables. One can argue that an FPGA is nothing more than a whole bunch of parallel pshufb units.

But that doesn't mean that two such tables solves all problems lol. It's often surprising how some problems blow up in your face and suddenly eat up thousands of LUTs.

Interesting i never thought of ASCII to be a table of tables I'll have another look into the ASCII table number sequences to better understand it. This thread is really full of gold nuggets.