I had a look in a Video Library - I found an example of an item lacking the Preview Image
- instead it just shows a folder icon.

When I checked in Windows Explorer I found there is no file inside the Preview Image
folder. Hence no thumbnail is displayed. You can manually add your own image to create a thumbnail.

I found a MS link on Video thumbnails and also a similar question posted here. According to this post you need to use Classic View. I'm not using SPO, I'm still on 2013, but hopefully something in this answer helps.