Basically, the steps you have to follow are (applies to sp2010, the might be slight differences in sp2013):
Find the id attribute of the ribbon button. eg. Ribbon.EditingTools.CPEditTab.Markup.Html.Menu.Html.EditSource-Menu16. Next go to 14 hive and under TEMPLATE\GLOBAL\XML\ find cmdui.xml file. This file contains the definition for all ribbon buttons in SharePoint. Search in this file by word Ribbon.EditingTools.CPEditTab.Markup.Html.Menu.Html.EditSource
Consider the Command attribute (which, in the example, is equal to EditSource)
in browser find this line and set a breakpoint, then you can step into to find the exact code that will be called (in the example, RTE.RichTextEditor.editSource() ). This is the handler
All you need to to is to pass some query strings to the aspx page without any JS call. As like in wiki this goes like this.
<a href="/demo/wiki/Seiten/home.aspx?**ControlMode=Edit**&**DisplayMode=Design**">Edit this page</a>