Even though MP4 is on the list of iTunes 11 supported video formats, sometimes, iTunes 11 fails to add and import some MP4 movie files, let along transfer those MP4 video files to iPod, iPhone, iPad, iPhone 5, iPod touch, iPad 3, iPhone 4S or Apple TV. Sometimes, you may run into such situation that iTunes does successfully import MP4 video files. However, it failed to sync the MP4 files to iPod, iPhone, iPad or Apple TV and only leave the message that one or more of the videos being synced from your iTunes library are not in the correct format to be played on your iPhone, iPad, or iPod.

TechiSky gives a detailed guide on how to solve can't add MP4 to iTunes 11 here:

http://www.techisky.com/how-to/solve-cant-add-video-mp4-mov-to-itunes-on-mac-or-pc.html

This guide also applies to import any format of video to iTunes 11 such as import AVI, MOV, M4V, WMV, MKV, FLV, MTS to iTunes 11