Like Eric Alexander said above, SharePoint creates a smattering of other 'helper' content for the videos such as renditions and thumbnail images. If you're only retrieving mp4 files, then you can use ContentType/Name = 'Video'.
If you will be retrieving other video file types (such as .mov or .avi), then you will have to get more creative because those come back as 'Image' content types. At first glance, it looks like SharePoint always uses .png to create thumbnail images, so you could potentially filter out results with content type 'Image' and FileLeafRef or FileRef values ending with '.png'.
Full disclosure: I haven't tested that thoroughly so use at your own risk.
Below is my complete api query:
[site]/_api/web/lists/GetByTitle('Videos')/items?$select=ContentType/Name,Id,Title,FileLeafRef,FileRef,OData__SourceUrl,VideoSetExternalLink,Created,Created_x0020_Date,AlternateThumbnailUrl,EncodedAbsWebImgUrl,EncodedAbsThumbnailUrl,FileDirRef,LinkFilename&$expand=ContentType