Which has higher density/is heavier - matcha liquid or milk?

I would say Milk !

Do you have a nice kitchen scale? Weigh 1 cup of each to see.

i remember getting a matcha milk tea, and for some reason it didn’t layer at all…

I would say that the specific gravity isn’t different enough for them to layer. Water and milk don’t really layer and your green tea drink is just a solute with its main component being water. If the green tea powder did something to the water, like turned it a bit gel-like, you might have a chance, but if the consistency stays the same of a hot chocolate, .... nah.

You could try to invent a layer drink by adding the smallest amount possible, though trial and error, of something like agar agar (a thickening agent made from a type of seaweed).

Have fun and happy food experimenting!