MSDN Library's snippet on EditFormUrl stipulates that one must use URL's that are relative to the root location of your content type. Personally, I've had no luck with that one.
However, the path that you specify in EditFormUrl can be relative to the root of your sharepoint instance; do be mindful of what you have put in your Schema.xml file, though.
Let's say I have a custom content type called "Lab Report". I've developed a custom form for it called EditLabReport.aspx. Rather than place this file in the _layouts folder, I've put it within my Lab Report content type "folder".
My Lab Report content type is used in a list called "Reports". In the Reports' schema.xml file, my section would look like this:
<Form Type="EditForm" Url="EditLabReport.aspx" SetupPath="features\$SharePoint.Feature.DeploymentPath$\LabReport\EditWorkOrder.aspx" WebPartZoneID="Main" UseLegacyForm="TRUE" />
Please note that, while the EditLabReport.aspx actually lives in the folder specified by SetupPath, it is called like this from your browser:
It is this URL that is to be specified in your EditFormUrl property.