Moving FTP Backup Files to SharePoint Online via Azure Logic Apps

Moving FTP Backup Files to SharePoint Online via Azure Logic Apps

I had the desire the other day to experiment with the Azure Web Jobs and see how they compare with some of the new functionality being rolled out. My idea was that I would take my site backups for the blog and copy them over to SharePoint for storage. Like many other techies, I host my own WordPress blog (which you’re reading hopefully). Luckily, I have the capability to generate periodic backups and I wanted to move these backups to a platform where I have more visibility. So, I though Azure Web Jobs would be the key to getting my datasets across to SharePoint Online. While this approach would have worked, when looking at the deployment options, I noticed that the available triggers and actions for the Azure Logic Apps have significantly increased. I already knew that it was easy to generate connections to SharePoint, however I was unsure if FTP connectivity was available. I dove in a little bit and noticed that not only is FTP available, SFTP (SSH) is available for Logic Apps. So now the “difficult” part, configuring the triggers and actions so my FTP backups would get copied. With a little bit of tinkering and luck, I was able to move the backups to SharePoint and I wanted to share that capability with anyone whom might need it.

So, if you have some files/backups available to you via FTP, here is the simple logic app to get your data moving to SharePoint Online. I assume at this point that you are familiar with the Azure Resource Manager, basic FTP functionality, and SharePoint Document Libraries.

First things first, let’s create a Logic App.

In the Azure Portal, add a new resource, and Logic Apps can be found under Enterprise Integration on the Marketplace menu

Give your Logic App a fancy name and reuse/create a resource group

And if all goes well, you’ll see the Logic App under your available resources

A quick note at this point, I chose the West US 2 region since it was close to the server being hosted. If you know where your server is hosted, the best practice recommendation is to create the resource in the closest data center.

Now, click into your newly created Logic App and you should be presented with a blank design surface option or one of many templates. In this case choose Blank Logic App.

We should have a blank template on which to create our new Logic Application.

Adding Some Logic

The first thing we need to do is got to the FTP Server and see what is in the directory, and if something has changed, we need to then trigger an action. So the first action we are going to insert in the blank design surface is the FTP action, if it is not one of the offered triggers by default, find it by simply typing FTP into the triggers and actions search box for the trigger to show up.

Another note, just because you choose FTP does not mean the connection will not use encryption. It specifies mainly the to be used by the API in the background. You’ll still be able to choose SSL encryption later.

On the FTP Configuration Screen, give the connection a name, and then specify the server details like URL/IP, Port, Username and Password. Then click Create.

You can see that there is still is an option to enable SSL even with the normal FTP protocol.

Use the FTP Folder Picker to choose the backup directory or the directory with the files you want to add. Make sure Include File content is set to true and then choose the interval you want your logic app to check the server. I chose three minutes so we’re not hammering the server checking for new files. This completely is dependent on your needs though. If you want to, this would be a great time to save your app before moving on to the next step. The save button is located on the toolbar in the top left corner.

Once saved, choose New Step and then Add an Action

In the list of actions, search for SharePoint and then choose create a file when it appears.

If you haven’t previously granted access to Azure to any SharePoint Sites, it will prompt you to allow the app to connect to SharePoint. So, click Sign in when prompted.

Once you sign in to Office 365 and grant the service access to your site, there should be a drop down where you choose your Site Collection and Subsite where you want to target the files. I chose my base Site Collection in the screenshot. In addition, “Folder Path” is the Document Library where you want to move the datasets, and it needs to be created in advance of the logic app executing for the first time.

The other two parts are inputs from the trigger, and will show as such in the dialog box when you click in File Name and File Content. Choose the same input options from the dialog on the right to keep the same filename and path. Alternatively, you could change the filename in an intermediate step, or depending on the filename, back it up to a different SharePoint site. However, we’re going to keep this example simple and only backup to one document library.

Now click save in the upper left and then click run to execute the logic app in a testing phase. Beware though, with FTP the trigger can run into situations with long running processes and get stuck. It may be better just to manually trigger the app in the Overview blade.

Once testing has begin, there will be a message that tells you to modify the content in your ftp server such as below.

Now, in order to see the application actually run, you need to put a file in the folder that the Logic App is checking. In my case, I just started another backup process that would place a new backup in the directory I specified.

Due to the interval that we set, we now have to wait and see that the process picks up the file from the directory, or that it doesn’t at all. Be careful that you don’t set the interval too long in the testing phase, since the logic app testing cycle has a timeout period. You’ll also see notifications like below, that demonstrate the app is “testing”

Once the application has successfully run, you should see your files begin to appear in the SharePoint Document Library

Now you can schedule this job to run however frequently you want, or you can always just trigger it manually in the portal if the files/backups aren’t automatically triggered.

Wrapping Up

The best part, is that not only can you create a file in SharePoint, you can target several other platforms to move the data. You can use Box, Dropbox, a File System, move it to another FTP, Google Drive, One Drive, or One Drive for business. If you have any other needs for platforms, you can create a custom activity and target the platform that you want.

The Azure Logic Apps platform is a powerful way to extend functionality without writing a ton of code. Logic Apps are flexible and practical, it makes Out of the Box configurations almost as powerful as custom coding. Please let me know what you think of this tutorial or if you have any questions on the steps within.

Leave a Reply

Your email address will not be published. Required fields are marked *