On the contrary, we could have no experience of the material world--it would not exist for us--unless we had a certain kind of mind, with certain concepts hard-wired into it. For example, time and space do not exist "out there," they are functions of the human mind.
I partly agree with this. Many of the details of the way that we perceive the world are contingent on the means that we have of interacting with it, i.e. the brain and nervous system. However, the brain itself has adapted over millenia in response to challenges posed by the world out there - albeit in a particular environment and on a particular scale.
Time exists in terms of thermodynamics 'out there' - (almost) the whole universe is a one-way street in this respect, and whilst our idea of space may only one amongst many possible ones, there would have to be some related concept for physical existance to make any sense.