I'm sure many people have thought about the behavior of first person mouse-look before, so how about a discussion? I had a lot of thoughts about advanced mouse-look models during last summer (2012). I thought I'd share my ideas. Please note that these ideas may be extended to any control schemes of first person viewing models (so, not limited to mouse-look control).

When I roll my eyeballs around, the motion I notice really doesn't seem anything at all like a spherical rotation being applied to a view tranformation. The motion seems more like my peripheral view is shifting. So it's like the projection matrix is merely offset. You can take D3DXMatrixPerspectiveOffCenterLH's functionality for concrete instance (just for a common reference, in case you really don't understand what I mean).

Obviously, any form of mouse-look interpretation which distinguishes movement between the eyeballs, head and the character person's entire torso/body will have a greater danger of being harmful to the gameplay experience of fast paced action games such as first-person shooters. I've been thinking on various ways by which mouse-look control may be interpeted most accurately by 3 domains of user-intention. If the player is walking or running, whole-torso movement twisting control will be most influenced by mouse-look activity (when running; this weight will be very high, 95 - 100%). Only when idle, if the player makes subtle/trivial movements with their mouse -- let's say within an 8 pixel radius in screen space -- the eyeball movement power may be concentrated even somewhete from 40 to 50% weight. Between the extremes of torso-twisting and acute eye movement, head movements can account for remaining behavior (though the weighting will be much nearer to eyeball movement than to the effect of torso twisting).

Also, the particular ways by which orientational components are affected through mouse-look can vary. Think of Euler angles. It's not very easy to 'pitch' or 'roll' your torso, though you may bend your back a little. The yaw of a person's torso usually will be a vector that is often nearly (or exactly) tangent to the ground which this entire person is standing upon. For heads, 'yaw' probably will need to be the most limited factor, though 'pitch' can have the most freedom while 'roll' is to have slightly less. For eyeballs, I'm guessing that 'roll' may only slightly get sheared when looking towards peripheral corners, but both 'yaw' and 'pitch' will probably be provided roughly the same degree of freedom.

Yet all of this mouse-look behavior is only determined on behalf of screen space-based mouse activity. What about a 'world space' interpretation? If the mouse movement vector -- a vector that is relatively designated as a screen-space offset located in the center of perspective which measures the horizontal and vertical deltas of mouse movement -- is ray-casted into the game scene, then what kind of influences should its sampling cause to the interpretation of mouse-look control?

...

I may easily write and speculate more on this, but I'm stopping here to get this topic posted and go to sleep.
So hearken, this community of might and glory; you geniuses out there! Please add your thoughts!

_________________
What most people don't understand is what I like to explain as a thing that I understand.

I've thought about this a little before and my main problem with trying to do anything fancy is that you ultimately render onto a screen which I must stare at fixedly with very little peripheral vision information coming from the screen. As soon as you start modelling eye-ball and head movements I think it's going to get really unpleasant to actually use.

My real eye-balls are in my head - are you really going to increase my immersion by modelling eye-ball effects in the controls and render?

You can mod depth-of-field effects into Skyrim. When I played with this enabled it was totally distracting. I think for any of these things to be worthwhile requires a display that covers the peripheral vision as well as a better input method than just mouse movements.

_________________
Whatever the mind can conceive and believe, it can achieve

Depth-of-field is usually poorly applied and over-exaggerated. Done right you shouldn't even notice a difference in gameplay, and it will really only be an aesthetic addition.

I understand your point, but I think that with careful research, rather than a senseless application, it can really add to the experience without disturbing the gameplay or anything else. That's what this topic is about... avoiding naive effects. I guess the best way is to experiment, but I created this topic to have a bucket out for anyone to throw in their ideas or remarks. Effects are applied naively all the time: bloom, lense-flares, fog (just as mist in a heavy jungle; it doesn't need to be a problem, but a lot of modern games apply it senselessly), SSAO, even normal mapping.


May you clarify that a little bit? I think I know what you mean, but I don't completely understand the mechanics of what you're describing.

_________________
What most people don't understand is what I like to explain as a thing that I understand.

I don't understand, will this kind of mouse-look change the experience you get as a player or is it just to enhance the graphical fidelity of a game?

_________________
Did you ever wonder, how time works?

Well, one of my friends mentioned how the real sense of inertia he felt while playing a first-person adventure game enhanced the experience. I think its mostly important to consider how you want the gameplay to feel. To the contrary, the game I've been planning to make for more than a year (only a concept at the moment), all of this kinetic phenomena is absolutely central to the gameplay. That's actually when I was originally thinking about this (during the summer of last year). In conclusion, it depends. Interesting though, eh? If you don't understand what I'm saying, then maybe you can clarify it from your own take and add to the discussion?

_________________
What most people don't understand is what I like to explain as a thing that I understand.

It's hard to define, but I remember in Gears of War, there was a feeling of 'weight' in the player's interactions with the environment. I guess it's a combination of carefully crafted sound, vibration and visual cues. Slamming into cover in GoW felt more 'real' than games like Halo3 where you glide and bounce around like a ping pong ball skipping across a hard surface.
IMO, this attention to detail creates the illusion that you're dealing with a physical world rather than a simulation.

BTW I'm not ragging on Halo, I don't want to start a holy war here.

_________________
10 PRINT "Bad Monkey ";
20 GOTO 10

Oh I see now, so its mostly about the feel so the games feel less floaty and more realistic. I think its a good idea then as lots of first person shooters feels really weird, like if the characters are kind of sliding around. I really like the idea of "peripheral view shifting" instead of a spherical movement of the eyeballs while only small movements are made. I have one concern tho, since the player already has the focus on the screen, wont it rapidly disadvantage the player while playing against AI driven opponents? Because if you apply realistic FoV in the game the peripheral vision will make it more difficult to actually see around you, which today action FPS needs.

so the question at hand is if the "enhanced realism" wont decrease the ability to play the game (unless its a horror game like deadspace, that might be fun )

_________________
Did you ever wonder, how time works?

In regular eyesight, I have a huge viewing angle and a lot of peripheral vision. The movements that my eyeballs make are usually very limited, say I'm walking along looking at shop windows - I move my head to face each display. The only large eye-ball movements are the initial ones that lock onto an item of interest from the peripheral vision is followed by a head movement to face that direction.

This is sounding to me like what you were describing (or at least what I was understanding). You have the eyeballs react first to the mouse by modifying the center of the projection and then follow that up with a simulated head movement. (I might be misunderstanding you, I have no idea how D3DXMatrixPerspectiveOffCenterLH changes the appearance of the render)

The problem here is that when looking at a screen I have a puny amount of peripheral information coming in, I can look at the whole screen without moving my head at all. So if I'm playing a game and I want to look somewhere else, in my mind I'm moving my head with my mouse and I do the eye-ball movement with my real eye-balls.

_________________
Whatever the mind can conceive and believe, it can achieve

Pitchfork is a better game than Halo.


Yep. Good point. Ultimately, I think the effect we desire most is a sense of attachment to a character's body, as Codehead described about Gears of War, as well as what Hazarth elaborated on.

_________________
What most people don't understand is what I like to explain as a thing that I understand.

I'd imagine it would get tiring to keep moving your head around, if we develop light-weight head mounted displays that offer a lot of peripheral vision. Maybe the paradigm would shift to the mouse controlling the torso/body rotation and then head movements could be controlled by the head. That sounds like it would get tiring, having to keep moving your head around in a fast-action game. Maybe just keeping the mouse control the same would be fine if the head rotation control was added on top of it. So in general, you walk around, looking with your mouse but if you have a natural head movement towards something, the display reacts accordingly. This could feel really weird though, the visual perception of head movements with no kinaesthetic feedback.

_________________
Whatever the mind can conceive and believe, it can achieve