How do I create a service in SharePoint Online like we used to do with timer jobs in on-premises?

Question

Requirement:

Given specific Sharepoint Online lists,

1. Create an app/service such that when triggered it will extract the SharePoint Online list data, transform to xml conforming to a given xsd and post it to an external ftp site

2. Schedule the above workflow at a certain frequency or manually invoke it.

License: Office 365 E3

What has been attempted so far:

For SharePoint on-premises, I would have easily used a timer job to accomplish this, but SharePoint Online does not support timer jobs. So the solutions that I have been able to extract so far with some of them eliminated are:

Solutions eliminated so far:

1. Azure WebJobs - I cannot use Azure WebJobs due to license restrictions. My org is not ready to get an Azure subscription.
2. A console app that consumes the SPOList api (and can be scheduled by Windows Scheduler) - the preference is to have the app within SharePoint Online and not outside

Solutions currently in play:

3. SharePoint Add-In (SharePoint hosted) - this restricts me to JavaScript. However even if I do manage to accomplish the above, will I be able to invoke this manually or on a frequency?
4. SharePoint Add-In (Provider hosted) - only allows me to create a ASP.NET web app but I need a service instead. Any ideas of how to create a service with an API that I can invoke on a frequency or manually?

My questions besides the ones on SharePoint Add-In:

1. Are there guidelines given anywhere (experience or MSDN) to not attempt a solution within SharePoint Online and rather go for Windows Scheduler?

2. Is it feasible to do this with SharePoint Add-Ins given the above questions?

3. Does posting to an external ftp site eliminate any of the above solutions?

Disclaimer: I am relatively new to SharePoint Online/Office 365, hence the above may appear a bit basic to some.

Answer 1

Any option you choose for scheduling (be it Azure WebJobs, which is a use-based cost model but no license required, Windows Task Scheduler, etc.) cannot run within SharePoint Online.

Microsoft has guidance on how to do this in the article Create remote timer jobs in SharePoint. The code can be run anywhere you wish, given you can somehow schedule the application you create.

Answer 2

Have you tried with MS-Flow ? You can easily extract the data from the SharePoint list and send it to other systems using relevant connectors.

Answer 3

As Trevor mentioned in the answer, the option lies outside of SharePoint.

Besides Azure Webjobs and Windows Scheduler, you can also checkout Azure functions. It is a compute resource , fashionably referred to nowadays as Serverless :)

When using Azure function, you have triggers that define when these functions should be executed i.e on a HTTP Request (HTTP Trigger) or after certain intervals (Timer trigger) or whenever a message is posted in Azure Storage Queue (Queue Trigger) etc.

You can decide what should happen after the event has occurred. So, you can run the CSOM C# code or a PowerShell script or even a JavaScript function based on your requirement.

Best thing about it is that it works on pay as you go model, so you only pay for the compute time (time taken to execute your task).

Limitation - Default timeout for an Azure Function was 5 minutes which can be increased to 10 mins. I think this will be more than enough for your tasks.

References - Azure functions

Azure functions as timer jobs in SPO

Calling SharePoint CSOM from Azure Functions

 Azure Function and SharePoint CSOM