You don't live in the real world.
The world that you experience every day is, not exactly a figment of your imagination, but a representation of the real world, constructed by your brain. To my mind the clearest example of this is your perception of colour – the sky is blue, but blueness doesn't exist in the real world. In the real world, light from the sky is composed of a wide range of different frequencies, from the ultraviolet band, through the visible and tailing off into the infrared. We know this because we can measure it instrumentally. Our instruments can measure any property we like about this light – in addition to its intensity over the wide range of frequencies, its polarisation and coherence (and probably other properties I haven't heard of). We can even measure it as individual photons. But we can't measure its blueness.
Of course we know that light over a particular range of wavelengths appears to us as blue, but in the world as measured by instruments, blue light is different to red light only in its wavelength – it is a difference of quantity, not of quality. The quality of blueness can't be measured, because it's a subjective experience inside our heads. The experience of blueness occurs when the receptors in our eyes which are more sensitive to the blue wavelength band, are stimulated more than the receptors that are sensitive to red and to green. The colour blue is something that belongs to our simulated world, not to the “real world” out there.
Now, the simulated world that we live in does bear some relationship to the real world, but not in the way you might think. The world that we experience is a greatly simplified representation of the real world, that helps us to survive. It contains the details of patterns that we are drawn to notice, many of which are hard-wired (is that a face? is is there a leopard hiding in the grass over there?) and some of which are learned. There is nothing that requires the simulated world in our heads to be accurate, foolproof, or even make us happy. Magicians actually rely on fooling our brains (this happens in entirely reliable and predictable ways) into constructing a world that doesn't correspond to physical reality, for dramatic and entertaining effect.
In fact, the world of our everyday experience is full of things that don't really exist at all. Take a teacup, for example – an object of some moderate significance in our world. You can do all kinds of things with it, it has properties and functions that are useful for you, you can describe what it's like including its weight, shape, colour, etc. But in the physical reality at the molecular and atomic level it doesn't even exist as an object. The aluminium, silicon and oxygen atoms that make the teacup don't care that they're part of a teacup, they don't even know that they are part of a teacup. Each atom is affected by electronic interactions with its nearest neighbours, but further away than that there is no physical, atomic significance whatsoever for the fact that they are part of a teacup, a dollar-shop garden gnome or a Ming dynasty vase. Now, this is the important bit: in our constructed worlds, the thing that for us is a teacup (or a gnome, or a vase), emerges from the relationships between the components at the level below. The teacup is not a real, physical thing, but a pattern that only exists, only is recognised, only has any meaning inside your head. It's just as Spoon Boy explained to Neo in The Matrix: there is no spoon.
This might seem silly, but it's even easier to explain with living organisms like yourself. Your physical body is composed of cells of many different types, which live, reproduce and die according to rules that keep your body functioning properly. Over 25 years or so, every single one of your cells will have been replaced – some of them many times over. You are now composed of different matter than you were 25 years ago. In fact your body is a self-sustaining pattern that continually absorbs atoms from the environment when it eats, drinks and inhales; and emits them continually through exhaling, sweating and various other bodily excretions. Yet somehow you are still you, a bit like the old joke about Granddad's favourite axe that has had its head replaced twice and the handle three or four times. The point here bears repeating: the things that are significant in the perceived world that we live in are patterns in the “real world” like a wave is a pattern in water. What you think of as the real world is in fact a representation of the relationships between the components of the real physical world, not the real physical world itself.
Now in esoteric thought, it is considered that we humans inhabit the physical plane, the lowest of planes. I wonder though, if the world of our everyday experience is actually a fairly abstract representation of the patterns in the physical world, the world of our experience isn't the lowest plane after all. The "real" physical plane of molecules and atoms exists below the plane that we live in. It isn't a direct part of our world at all, but it's required absolutely for our world to exist. The patterns of our world are patterns in something – the physical world of atoms and molecules.
Even atoms though, are really just patterns of organisation of subatomic particles with emergent properties of their own. The protons, neutrons and electrons in the silicon atom in your teacup are the same as those in the aluminium atoms – they are merely taking part in a different dance and forming a different pattern with different emergent behaviours. At this level and below the world is a weird mix of quantum mechanics and strings vibrating in eleven dimensions: a world that is utterly unlike the world of our experience, dimly and imperfectly glimpsed through the lens of mathematics. But real, nonetheless. If it weren't so you wouldn't be reading this, because despite being utterly alien to us, quantum mechanics works and is the basis for the electronics we use every day.
What about the other direction though? If the patterns that make up our world are patterns in the plane below, what of the patterns in our world? If the constructed world we live in is a representation of patterns in atoms, which themselves are patterns of subatomic particles, which are in turn patterns in vibrating string-stuff, what is the significance of the patterns in our world? Do they form a constructed reality for consciousnesses inhabiting the plane above? And what of the patterns of those patterns, at some even higher level? Scientific thought largely ignores this. In fact, science goes to a great deal of trouble to exclude any patterns at all from outside the experiment, to make clear the very simple kinds of interactions that are within our intellectual capabilities. This is a very good reason, but I think it has left a kind of cultural blind spot that comes with scientific training. The scientific study of the patterns themselves – ecology, chaos theory, whole systems analysis, are fiendishly complex and even then are not predictive of the emergent behaviours at the next level. They study the interactions, not the meaning of the interactions in the next plane up.
So in the prevailing western scientific world-view, perhaps as a result of this cultural blind spot, the assumption is that there is no significance at all to patterns in our world – except for the special case of the patterns of neuronal organisation in our brains. These special patterns somehow have the emergent property of consciousness (and among other things, the subjective experience of the colour blue). I suspect that the only reason for this acceptance is that it's difficult to deny the reality of subjective experience, and without a non-physical soul or spirit or some other essence (an unpopular notion in science these days), there doesn't seem to be any other way to explain it. It must therefore be an emergent property. So we're left with the slightly absurd notion that the complex network of interacting components in a human brain gives rise to consciousness (and, grudgingly, perhaps this is also true but to a lesser extent in animal brains), but that other kinds of complex networks of interacting components do not give rise to consciousness. This notion is of course entirely untestable in any way that is acceptable to science and hence is not properly a part of science at all. Ecosystems, human societies, weather patterns, biospheres, all are assumed to not give rise to consciousness in the same way that a brain does. There's no reason to make this assumption, in fact it may be more reasonable (and perhaps safer!) to assume the converse.
On the other hand, if consciousness is not “just” an emergent property of sufficiently complex physical systems (or whatever property it is that does the trick), and it arises because of some other factor like a life force or soul, then why wouldn't that factor also manifest in ways other than human consciousness? Unless your religion insists that said factor only applies to humans, but this doesn't seem any more satisfactory than the special pleading in the scientific world-view that only brains, and not any other kind of complex interacting network, cause the subjective experiences of consciousness to arise.
Either way, those subjective experiences do arise. You and I both do perceive the colour blue, and both recognise a teacup when we see one. Whether my experience of blueness is anything like yours, I have no idea. I guess that it probably is, because we're both humans with a lot of biology in common, but this is only and always will be just a working assumption. Although our experiences may be similar to each other's, they are not in any way similar to what “the real world” (whatever that might mean) is really like.
It could be that the world and its planes are just patterns all the way down, and may well be patterns all the way up as well.