You, Castle does not say, are dumb as a bag of hammers.

"Yeah. This place exists outside of time and space, so when people come here, they come from all different times. You're obviously from a little bit further in the past than your sister and me."