top of page

How to Move 400 TB of File System Data into SharePoint Online


Several years ago, while exhibiting at a Microsoft conference, a surprising trend began to show itself (at least to me…). Organizations of all sizes were looking for lower-cost SharePoint Online extended storage because they were running up against their allocated SharePoint Online storage limit – which was then and still is now, 1 TB plus 10 GB for each active license.


In fact, the SharePoint Online storage resources available to subscribing organizations are based on the organization’s type of SharePoint Online or Microsoft 365 license.


As stated above, by default, Microsoft provides 1TB of storage for a SharePoint Online subscription plus 10 GB for every purchased license. You can buy additional storage from Microsoft if you need more space apart from your subscription-allocated storage resources. Whichever license plan your organization has, the total “cannot exceed” storage limit for a SharePoint Online site collection is 25TB.

For example, a company with 900 Microsoft 365 E3 licenses would have a total of 10 TB of allocated SharePoint Online storage available: 1 TB + (900 x 10 GB). 10 GB of storage might seem like a lot, but with the file types now being created, received, and stored, such as Word files, PowerPoint files, Excel files, PDFs, TIFF’s, JPEGs, and video files, storage space is consumed quickly.


In that same example (see Figure 1 below), the organization could purchase an additional 15 TB of SharePoint Online storage for $0.20 per GB per month or $36,000 per year. Microsoft tells SharePoint subscribers they must delete legacy SharePoint files once the 25 TB is reached. When asked about going beyond the 25 TB maximum storage capacity, Microsoft says they must purchase a third-party SharePoint extension/archiving solution.






Figure 1: SharePoint Online offers few answers to adding additional storage. When you reach your organization’s allocation limit, you must delete files or purchase extra expensive SharePoint Online storage from Microsoft







However, adding a separate, cloud-based archiving platform to offload legacy inactive data from an enterprise SharePoint instance is a viable stop-gap measure but adds additional complexity and cost.


To be clear, there are two challenges with SharePoint Online storage:

  1. Expanding SharePoint Online storage beyond your corporate allocation for day-to-day employee activities, i.e., uploading and revising daily documents

  2. Uploading large file sets, for example, whole file shares, into SharePoint Online for ongoing information management activities. Many SharePoint admins are working with others in IT to “move to the cloud.” However, several hundred TB on-prem file shares have become a massive roadblock for this move to the cloud

Looking at the first challenge above – extending SharePoint storage beyond the maximum limit is the answer and can be done utilizing Offload Data Virtualization technology. We will discuss this capability in the next blog in this series.


Let’s look at the second SharePoint Online storage challenge – uploading legacy files into SharePoint Online beyond the maximum storage limit.


Instead of deleting large numbers of potentially valuable files or incurring the complexity and cost of a separate archiving platform, many IT professionals are now asking for/considering a long-standing and proven storage solution called Copy Data Virtualization or CDV.


What is Copy Data Virtualization?

Copy Data Virtualization (CDV) is a technology that creates and consolidates enterprise data into a logical view from multiple storage repositories without needing to move the data physically. This allows businesses to access and manage their data more efficiently without worrying about the underlying complexity of the enterprise data sources.


CDV has been used for a variety of purposes, including:

  • Data integration: CDV can integrate data from disparate sources, such as on-premises databases, cloud-based data lakes, and SaaS applications. This can help businesses to consolidate their data and gain a single view of their operations

  • Data replication: CDV can be used to create copies of data for backup, disaster recovery, and test/dev purposes. This can help businesses protect their data and ensure they can always access it when needed

  • Data masking: CDV can mask sensitive data, such as customer PII, for security purposes. This can help businesses to protect their data from unauthorized access

  • Data governance: CDV can help businesses manage their data assets. This can help businesses to ensure that their data is accurate, consistent, and compliant with regulations

CDV is a powerful capability that can help businesses improve their data management practices while consolidating large amounts of data into less expensive storage resources. It can help companies to reduce the complexity of their data infrastructure, improve the agility of their data processes, and improve their data security.


Copy Data Virtualization is a proven option to improve your data management practices for both on-prem/cloud and cloud/cloud, to increase employee productivity, and reduce overall enterprise cost and complexity.


With that general background in Copy Data Virtualization, how can it address the second SharePoint Online storage challenge described earlier?


Copy Data Virtualization for SharePoint Online

I have discussed the benefits of storage consolidation and virtualization in past blogs, but how can you address the SharePoint Online storage issues of the high cost of purchasing additional SharePoint storage from Microsoft, not to mention the 25 TB maximum limit?

As I covered in the previous section, Copy Data Virtualization (CDV) is a data virtualization technology that creates and consolidates a logical view of data from multiple sources without needing to move the data physically. This allows businesses to treat all storage media in the enterprise as a single pool of storage that employees can access easily and utilize quickly without worrying about the underlying complexity of the data enterprise data map.


With SharePoint Online, organizations can migrate huge data stores into SharePoint and never worry about their storage limit. Your SharePoint instance would look like it was managing 100, 200, 400 TB, or more, all while adhering to your license allocations.


SharePoint Online has become a widely popular solution for many years. However, SharePoint Online storage limits have also been a long-standing complaint for users and admins. Storage virtualization is a straightforward and efficient way to address this long-term challenge. However, several vendors have offered an “extension” solution that acts on existing SharePoint data after storing it in SharePoint. However, this does not easily address the issue of migrating massive amounts of files into SharePoint for ongoing management. Using the extension method for SharePoint, admins would need to migrate files into SharePoint slowly in smaller amounts while manually extending files out. This manual process could take quite a bit of time.


But what if you could upload 400 TB of files into a secure cloud and then provide virtual pointers to your SharePoint Online instance after the fact – all in one process? This more common-sense strategy would save time and money while ensuring little or no impact on employees.


The restorVault Copy Data Virtualization solution allows you to do just that!


restorVault Copy Data Virtualization

As I mentioned at the beginning of this blog, there are two aspects of SharePoint Online storage allocation and data management:

  1. How to import massive existing data sets, for example, the 400 TB from an on-prem file server into SharePoint Online (even though the maximum limit for SharePoint storage is 25 TB)

  2. And how the business data added to SharePoint daily will affect the storage limit

Both business needs will count towards the SharePoint Online storage limits. The balance of this blog will focus on how to import giant data sets into SharePoint without exceeding your storage allocation.


Note: A future blog will discuss extending SharePoint files into a separate low-cost cloud platform while keeping the extended files visible and accessible from within SharePoint.


restorVault CDV provides for fast replication and restoration of files in the SharePoint Online cloud through the use of Virtual Data Files. With each VDF occupying only 1K in size, you can extend SharePoint Online storage with VDFs in just a few minutes while the “golden copy” (copy of record) remains safely protected (encrypted) in immutable data vaults in the separate restorVault cloud.


restorVault’s Copy Data Virtualization for SharePoint migration:


  1. Allows the migration of huge amounts of files into SharePoint without exceeding the storage allocation

  2. Uses up to 80% less SharePoint Online storage

  3. Provides for swift file restores (to somewhere else if needed) if SharePoint Online suffers issues

  4. Provides higher levels of data security against ransomware and extortionware

  5. Addresses one of the most significant issues storage administrators have with SharePoint: what to do with your existing huge on-prem file stores


Figure 2: restorVault’s Copy Data Virtualization solution is for companies that have run up against their SharePoint Online storage allocation


Revisiting the central question of this blog: how can you migrate 400 TB into SharePoint Online without worrying about your maximum storage allocation? Unlike any other solution available today that infers it can address this need - restorVault’s CDV solution actually can!


Blowing through your SharePoint Storage Limit

restorVault’s patented Copy Data Virtualization ensures you can migrate hundreds of TBs of existing files into the SharePoint Online management sphere without running up against the annoying SharePoint storage limit or being forced to pay the inflated SharePoint Online storage prices if you need to extend SharePoint storage.


restorVault’s patented CDV solution accomplishes this by migrating the large file sets into restorVault’s secure and immutable Compliant Cloud Archive (CCA). We then set up an Azure file server where we push the data’s Virtual Data Files (Pointers) from the files located in the restorVault secure cloud down to the Azure file server. In this way, SharePoint sees the pointers in the Azure file server as the original files located in SharePoint.


restorVault SharePoint Online storage extension solution using the Compliant Cloud Archive and Copy Data Virtualization technology provides several benefits for administrators and end users:

  • Huge data sets can be uploaded and seen by SharePoint as standard storage resources

  • The files extended from the restorVault cloud to the Azure file system, and SharePoint are fully manageable by SharePoint

  • The restorVault Cloud stores two copies of the migrated data

  • All files stored in CCA are encrypted and stored on the immutable storage tier in the restorVault cloud in case of a ransomware attack

  • In case of data corruption of ransomware attack, the file pointers can be re-pushed down to the Azure file server in minutes, ensuring access to files for the employees


SharePoint sees the Azure file server as an extended storage resource but is not classified as SharePoint storage and is, therefore, not subject to your SharePoint storage allocation.


restorVault’s Copy Data Virtualization solution provides a refined solution to safely store your inactive or semi-active unstructured data and other high-value unstructured data safely and inexpensively in a trusted and immutable cloud vault while presenting a consolidated view for easy access and management.


Other restorVault Solutions

restorVault’s Compliant Cloud Archive (CCA) provides a secure, virtualized cloud storage platform for on-prem and cloud-based file shares for long-term information management, on-demand access to virtualized unstructured data, and the highest levels of data security, including an option to store your data in an immutable cloud storage tier for ransomware/extortionware protection.

The Tamperproof Cloud Storage solution (TCS) provides a hot standby-like protected storage repository that allows for complete disaster or ransomware recovery in minutes, not days.


Offload Data Virtualization (ODV) ODV frees up precious capacity on primary servers by offloading inactive data to a protected CCA or TCS vault based on policies. ODV can also provide SharePoint Online live data extension for live day-to-day file movement out of SharePoint based on policies, etc.

  • Only active files stay local

  • Conserves primary capacity

  • Allows on-demand access

  • Prolongs system lifespan


Please reach out to us today to learn how restorVault can help your company save money while increasing data security and storage capacity.


Комментарии


bottom of page