Engine Works
Under the hood of Alteryx: tips, tricks and how-tos.
So you just created a macro? Fantastic! Macros can save Alteryx developers tons of time and organizations tons of money. Before we start deploying our macros to production workflows, as developers, there are a few questions we need to answer, such as: Where will the macros be stored so that all developers can access them, or will these macros need to be updated in the future and will that impact all the workflows they exist in?
The intention of this blog is to explore different options for deploying macros to your Alteryx environment, both in the Alteryx Gallery and on local machines. No option is better than another, it will just depend on the needs and practices of your organization. By stopping and addressing these important questions before you start deploying macros, you could save your organization from a significant amount of problems in the future!
Important Considerations
How an organization deploys and maintains macros depends on a range of factors, such as who will be using the macros, the frequency at which they are updated, etc.… Understanding an organization's primary considerations is important before exploring different ways to deploy macros.
One of the first questions you should be asking yourself is, what will be the update frequency of the macro? Within this seemingly simple question are two sub-points that you need to consider: macro versioning and making access dynamic.
As the software development life cycle process becomes more common within organizations that use Alteryx, it is important to consider how you will be tracking the different versions/changes of the macro. Will all versions of a given macro continue to be stored in a repository, or is the plan to overwrite the macro every time a small change is made? And what about dynamic access? Is the intention to package the macros in each workflow and make them inaccessible to other users, or would it be better to set up a repository for your macros to exist in, not only for your workflows to access but for other developers? (Probably the latter!)
Figure 1: Many organizations are creating workflow / macro repositories for tracking the SDLC process
A second important consideration to make before deploying macros is storage preferences. This is highly dependent on the practices of the organization you are working within, but it is important to note that there is an ongoing discussion about where macros should be stored.
Alteryx itself recommends using a shared drive for dynamic access, but there also exists a growing preference to shift towards storing items within the cloud, whether that be SharePoint or OneDrive, as opposed to a local or shared drive. All these options will be explored in this blog, and we will be able to compare the pros and cons of each one. Part of this exploration will require lightly touching on related issues such as: storing different versions, which options allow for dynamic access, and creating macro repositories.
Figure 2: The amount of companies using cloud storage has rapidly increased over the last decade (Source: Statista).
While it may not be relevant to every organization, your plan for deploying macros may want to consider the SDLC process that is becoming more commonly used in the development of Alteryx tools. It is important to think about how the organization will store macros in different repositories, how macros are deployed to a production environment, and the implications behind updating a macro and moving it through the SDLC cycle again.
Finally, make sure you are thinking about the future goals and plans of your organization as it relates to Alteryx or even just Information Technology in general. There is an understanding that future enhancements to Alteryx, such as the increased usage of Gallery APIs or an organization's desire to fully develop their SDLC practices, may impact how an organization wants to handle macro, or even workflow, development.
One primary concern when planning out this process is how the foundational deployment can be used to ease a potential future transition to a more automated process. Specifically, an organization should design its macro deployment to make it easy to use XML of macros to update versioning/track changes, push macros automatically from one repository to another, and utilize APIs to automate as many processes as possible.
Deployment Best Practices
Before we discuss a few different ways to approach deploying macros to your Alteryx environment, it will be good to have a high-level understanding of industry best practices. I have been fortunate enough to chat with multiple Alteryx experts within my organization while also researching online to better understand how different businesses handle macro deployment. Please note I am not suggesting that this is how macro deployment should be done. Rather, having a high-level understanding of industry best practices will be helpful to an organization before different options are explored.
How a business deploys macros/workflows is highly dependent on a variety of factors, such as how many people use Alteryx within the business, how the business commonly stores files, and whether the macros need to be updated frequently after being deployed. Here is a quick synopsis of best practices:
Gallery
Many businesses that deploy macros do not intend on updating them or making changes to them once they exist in Gallery very frequently. As a result, the macros tend to exist within packaged workflows when moved to Gallery. Being packaged, the macros are static, but when an Alteryx user pulls it to their local machine, they have access to the macro and can open it and change it as needed. Macros, which themselves are workflows, can be uploaded to Gallery so that users can pull them to their local machine and use them when needed. Organizations will, at times, store the macros in a collection so that users can download them and implement them in their workflows.
Figure 3: We can store macros in the Gallery but cannot reference them in workflows
Shared Drives
For businesses that don't want static macros, the most common practice is to use shared drives. There are multiple reasons why storing macros in a shared drive is beneficial, with the main one being that everyone can easily access the macros. The individual users on local machines don't have to download a package of macros or pull them from the Gallery to access them. They can just point their Designer to the shared drive and will have access to the most updated macros.
Another advantage of the shared drive is if the organization has created a separate macro section in the tool palette and pointed the macro repository to the shared drive, then whenever a macro is added to the shared drive, it will exist within their local Designer.
The final advantage is that macros can be easily updated. When uploading a workflow to Gallery by default, the macro is stored within a packaged workflow, but there are alternatives to this. A user can instead identify a folder path to the macro. When this is done, every time a workflow is run in Gallery, it looks to the path to execute the macro part of the workflow. If the path is a shared drive, then when the macro gets updated within the shared folder, the workflows that use the macro will run on Gallery using the updated macro. With an understanding of the primary considerations and industry best practices, we can finally cover a few different approaches to deploying macros!
Exploring Approaches
The first option to be explored is to deploy your macros using just the Gallery or Alteryx Server. This approach would entail creating collections to store macros in the gallery. Users would be able to access the production macro collection and download the macros to their local machines. Since workflows in the Gallery with macros in them cannot reference macros that exist within the Gallery, the macros would have to be packaged with the workflow when deployed to the Gallery. Once macros were downloaded to the local machine, they could be placed in a specific folder that could be referenced within Alteryx to create a tool palette of macros.
While it is good to know this option exists, it is generally not conducive to the previously discussed primary considerations. For one, it is not easy to update macros. Any workflows that would require the updated macro would have to be pulled down from the Gallery, updated, and republished. It would be easy enough to store macros in different collections and move them through SDLC, but versioning would likely have to happen using the Gallery’s versioning system. With everything existing in the Gallery, it would be very possible to use the Alteryx API and other features to move macros to new collections and store different versions, but it would likely be a lot of extra steps to build out.
Figure 4: Versioning for workflows or macros can happen within the Gallery itself
Approach two would deploy macros to the shared drive for all users to access. The main benefits of this are that any Alteryx user could access the production macros from the shared drive, and the Gallery itself could reference the newest versions of the macro. Users would not have to open workflows and switch out the macros when they're updated. Users could also reference the production macros folder in their Designer to get a tool palette of all the updated macros. While this approach may not align with many organizations’ long-term goals of shifting to cloud storage, the benefit is that users would not have to store anything locally to use and deploy the macros. As it relates to SDLC, macros could be deployed to collections in Gallery, but the production macro would still have to exist in a shared drive. Macros in a theoretical Production collection in Gallery would never be referenced in workflows (this isn’t possible with Alteryx) but would still be good for tracking workflows through SDLC. As far as future processes, it would be easy to automate the movement of macros from one collection to another as it goes through SDLC.
A third common option is to deploy macros using Gallery and SharePoint. There are a few things to note about SharePoint before your organization considers using it to deploy macros. One is that there are two types of SharePoint: SharePoint Online and SharePoint On-Premises. SharePoint Online has file paths represented by URLS, and everything exists within the cloud. SharePoint On-Premises or SharePoint Server has the organization install and manage the infrastructure themselves. With On-Premises users "access files and resources by connecting to the local SharePoint server or network location using file paths specific to that environment." I am going to operate under the assumption that most organizations are using SharePoint Online.
The second thing to note is that Alteryx has tools that can connect to and output to SharePoint. That said, it seems to be limited to common files for storing data/tables and is not relevant to deploying macros. Without the introduction of other tools and features like OneDrive and Sync so that one can create actual file paths, it doesn't seem possible that macros can be stored in SharePoint and reference said macros to be used within workflows that are deployed in Gallery. The reason for this is that on the machine hosting Alteryx Server, there needs to be a file path to macros that are being used in Gallery. If there is no file path, then the macros must be packaged within the workflows. SharePoint could still be used to store production macros that users could pull down to their local machines, but it does not seem like it would be an effective way to dynamically deploy macros to Gallery without the inclusion of OneDrive to allow for the creation of folder paths. If the organization is okay with deploying static, packaged macros, then this approach will work great! It allows for cloud storage of all the macros, and production macros could easily be pulled from SharePoint for users to deploy to their local designer. If you want to deploy dynamic macros using SharePoint, then you must use it alongside OneDrive.
If SharePoint is going to be explored as an option, then we must equally consider our fourth and final option: OneDrive. OneDrive differs from SharePoint in a few ways. One is that it is primarily designed for personal storage for individual users. Two is that although it stores files in the cloud, users can create files on their local machines that can be synced to the cloud and shared with others. Since OneDrive has actual file paths, it is possible to share the relevant folders with the machine hosting the server to store macros in OneDrive and dynamically deploy them to workflows in Gallery. The process would be very similar to the approach outlined in the shared drive section, with the exception that OneDrive folders are stored in the cloud (even if you can see them in your local file explorer).
There are some caveats to this approach. One is that OneDrive folders would still have to exist on the machine hosting the server and on the local machines. This is because we still need a file path for the Gallery to reference. Two is that any folder that stores production macros for deployment would have to be shared with all the users who may need to use the macros (as well as the machine hosting the server). As touched on earlier, using Microsoft's Sync feature would allow an organization to use SharePoint in conjunction with OneDrive to allow for cloud storage while still having actual folder paths, but further exploration of the viability of this approach as it relates to server deployment is still required.
|
Method |
Pros |
Cons |
|
Gallery |
· Easy to track versioning · Macros are stored in the cloud · Conducive to future automation |
· Macros must be packaged and static · Difficult to update macros in existing workflows |
|
Shared Drives |
· Easily accessible to developers on local machines · Macros can dynamically update in Gallery workflows |
· Doesn’t use cloud storage · Requires setting up a system for versioning |
|
SharePoint |
· Macros are stored in the cloud · Dynamically updates macros in Gallery workflows · Easily accessible to developers on local machines · Conducive to future automation |
· Requires using OneDrive to create file paths if you want dynamic macros · Requires setting up a system for versioning |
|
OneDrive |
· Macros are stored in the cloud · Dynamically updates macros in Gallery workflows · Accessible to developers if the files are shared · Conducive to future automation |
· Requires sharing of folders · Requires setting up a system for versioning · Limited online interface given the primary function of OneDrive |
Conclusion
How an organization deploys and stores macros is dependent on a range of factors. As an Alteryx Developer or Administrator, it is important that you understand these factors before developing a strategy for deployment. In this post, we have explored a few ways that macros can be deployed in an organization, but these are not the only ways to approach this unique problem. Consult with members of your organization and the Alteryx community itself before executing your plan, and make sure to share your approach on the Alteryx forums so that we can all continue to learn and grow together!
Here are some helpful resources for determining how to approach the deployment of macros in your organization:
Creating Your Own Tool Category in the Tool Palette
Sharing Macros in Alteryx Designer and Server
