He did not notice he hit her with that last toss. He does notice the sigh, but it is still a moment before he is sufficiently roused to turn and find the source of it. By that time, the paper has rolled to a stop, and he does not see it.

And so, he just glares at her noise for interrupting him, and returns to his work.