Have you ever heard the expression 'a few bricks missing from a building?'

You mean "a few BRICKS missing".

I haven't heard that one but it's in line with all those expressions like "a sandwich short of a picnic" that just mean somebody is a bit doolally.

Someone who lacks both common sense and book smarts. Usually reffering to someone who does or says something ignorant but sometimes it also can be about someone who is silly or goofy.

At a construction site.