Found a hacky way, using substring of FileRef:

    /sites/mysite/_api/web/lists/getbytitle('Useful Links')/items?$filter=substringof('Useful Links/My Folder/', FileRef)