Does your business rely on users being productive with software that loads projects at a time wherein many linked and embedded files open at once? And do these apps become unstable with latent file access? If so, finding the right way to leverage cloud storage is going to require careful planning.
Many cloud storage options available but few solutions specifically geared towards engineering and creative apps
The status quo, of course, is to keep piling up data on traditional filers like Dell EMC Isilon, NetApp, or perhaps your Windows Server clusters.
There are numerous consumer cloud storage options, but we are talking enterprise here.
Enterprise cloud storage and collaboration options such as OneDrive for Business, SharePoint Online, Box, Slack, and many others are not sufficient because they have too many arbitrary limitations (e.g., maximum file size), or they are not designed for the workload.
Vendors like Nasuni offer a way to get all your users on the same namespace, across locations, and elegantly leverage the cloud for tiering and backup without forcing a significant change on your users. However, not many organizations can support the price tag of Nasuni, nor do they find a wholesale replacement of their storage infrastructure appealing.
Azure File Sync from Microsoft could be a solution, but it does not have global file locking, and the numerous limitations and costs tend to make it a non-option for larger organizations.
Why not a cloud file share?
IT professionals often search for a cloud file server. Storage that behaves precisely like network-attached storage (NAS) devices on-premises, but sits in the cloud.
There are a few problems that make this a non-viable option.
First, it does exist in the cloud the same as it would in your datacenter. From a networking perspective, it isn’t a domain-joined appliance, and you are probably not going to undertake the security exercise to make it so.
Secondly, and perhaps the biggest issue is that your users and apps reside elsewhere. And these apps, typically running on workstations, do not like latency when they open a project.
Cloud storage gateways to the rescue?
The idea has been to plunk storage appliances down near the users so that local copies of the data can exist to avoid problems with apps that choke up on latent file access.
The problem with this approach is that the solution is cumbersome and expensive. Consider that most customers already have on-premises storage infrastructure investments. And the cost of the cloud storage gateway is typically outside of most IT budgets.
Let’s face the facts: Archiving 101 is the answer
A cloud-based file share is either impractical or too expensive, and most cloud-based collaboration subscription software is not practical either. So, does this mean that cloud storage is out of reach for engineering and creative companies?
Enter file archiving.
As is typical, when solving complex problems, a slight change in our perspective can be necessary to discover the solution.
In this context, the problem seems to be without a solution when our perspective is towards finding a like-for-like replacement of traditional NAS for something low cost and cloud-based without any change for the users and apps.
The fact is that, unless you are willing to pay premium amounts for a solution, the technology is not there.
However, what if we recalibrated our perspective?
Instead of perpetuating the status quo of hoarding all data on expensive primary and secondary file storage arrays with users having immediate access to it all, we could shift our thinking to a data management strategy that unlocks the benefits of the cloud with minimal user behavior change.
Why file archiving makes sense for engineering firms and creative companies
Engineering firms and creative companies typically store data for long periods. And it is usually entire projects that, when complete, see very little—if any—retrieval activity of the data.
Given that access is so infrequent once a project is complete, it makes sense to think of the cloud as an archiving destination as opposed to a primary storage solution. Conversely, archiving is no longer a folder copy to a secondary, disk-based storage array with users expecting instant access in the rare time they need access.
With HubStor, the cloud is at the fingertips of your users. They can peruse file directories, retrieve files, or recall entire project folders. You can provide a full-text search experience also.
Users will need to adjust to surfing a web-based interface and pull down the files/folders they need. And HubStor enables IT professionals to control the available on-premises storage locations to which one can rehydrate folders from the cloud archive.
In case you are thinking the disruption to your users is too much to ask, consider the economic advantage. We can now store one petabyte of data in the cloud for less than $30,000 annually, assuming the use of the archive storage tier with Azure Storage Reserved Capacity. And for an additional $10,000 annually, you have a reliable cloud backup of this same data volume. These figures include HubStor and Azure costs combined.
With HubStor’s inline compression and deduplication, you might get 1.2 to 1.4 PB mileage out of that budget amount.
When we talk to large engineering and marketing firms that are accustomed to keeping all data on expensive storage arrays, discovering that they can store data for this low cost while maintaining simple access for users is eye-popping! It seems the incumbent storage oligarchs have done an excellent job at marketing over the years while lagging behind in innovation, but times are changing.
File archiving best practices
HubStor delivers a broad spectrum of file system archiving options and policy controls.
In general, simplicity is a good rule of thumb. Here are some general best practices for file archiving:
- Start small; target low-hanging fruit to begin your archival processes – Don’t try to boil the ocean. Get some easy wins under your belt and expand from there. Do you have data sets that nobody would notice if they were gone? Yes? Great. Start by archiving this and then introduce your users to the concept gradually from there.
- Leverage last accessed date in your policies – File systems keep a record of the last accessed date. With HubStor, you can leverage this in your strategies, such as targeting files last accessed beyond a specified time only.
- Consider file type-based rules also – Maybe you have specific file types that are more or less ideal to archive. For example, large files—especially low-touch ones—are top candidates for archival because they’ll give you the most significant storage reduction.
- Take advantage of link-based sharing and search – In HubStor’s user portal, you can provide users with link-based sharing so that files do not have to be downloaded and transferred by other means. Additionally, the search experience can help your users to find what they are looking for faster. Conveniences like this will help people embrace the new archiving paradigm.
If you need to map out and determine what kind of data is ready to be archived based on the criteria above, we’ve created an application which will help you do just that. File Storage Analyzer will show you whats hiding in your data and which files are ripe for archiving.
Comments on the user experience
Leveraging the archive tier in the cloud is compelling because of the significant reduction in storage expenditure. However, as an IT professional, you’ll need to adopt it carefully or risk backlash from users.
The primary consideration is that files on the archive tier do not rehydrate instantly. Instead, they can take several hours before they are accessible.
HubStor offers policy controls so that you can model and isolate the data you wish to move to the archive storage tier in the cloud.
And HubStor providers users with a simple experience for recalling items from archive storage. Users can still navigate data residing in archive storage—even finding them in searches—and, upon a retrieval request, the user will get the notification that the item is on the archive tier. If they proceed with recalling it, they can subscribe to an email notification that delivers a link for fast retrieval when it is ready.
What about storage tiering?
Indeed, HubStor includes storage tiering options that deliver a software-based cloud storage gateway on your existing storage infrastructure.
Tiering works most elegantly when there is a Windows Server front end to the storage architecture since seamless recall can occur transparently to most apps. However, it can work with NFS and straight CIFS shares also.
There is no reason you can’t leverage HubStor’s cloud tiering capabilities as part of your archiving strategy. Most HubStor customers do precisely this. Just keep in mind that some apps do not like slow file retrieval, especially when it comes to project data sets that recall numerous linked and embedded files. Using HubStor’s folder rules and file type clauses, you can target cloud tiering to the specific directories and file extensions that make the most sense for your organization.
HubStor has a growing list of AEC and marketing/creative companies, many large enterprises with enormous data volumes. If you are considering your cloud storage options and see if HubStor might work for you, please reach out to speak with a technologist.