This is not possible using SharePoint out of the box capabilities.
You have to break the item permissions & then grant necessary permissions on list item.
You can achieve this using SharePoint designer 2013 workflow which will run on item creation/update & perform above operations.
Check this documentation for 2013 workflow actions: Workflow actions ...
Per my test, it works well, please fill in the user email via Enter custom value:
If the issue still exists, we suggest you create a new task list, create a new flow to create task in this list to compare the result.