Storage migration is one of the most challenging tasks in data center operations. How do you move terabytes of data from old arrays to new ones while keeping applications running? FC-Redirect provides capabilities for this, and I’ve learned a lot about what works and what doesn’t.
Why Migrate Storage?
Storage migrations are necessary for several reasons:
Technology Refresh: Storage arrays have a limited lifespan. After 3-5 years, they’re out of warranty and support, making migration necessary.
Consolidation: Companies merge or consolidate data centers, requiring data movement.
Performance Upgrade: Moving from older, slower arrays to newer, faster ones.
Cost Optimization: Moving less-critical data to cheaper storage tiers.
Data Center Relocation: Physical moves require storage migration.
The challenge is accomplishing these goals without disrupting business operations.
Migration Approaches
There are several fundamental approaches to storage migration:
Host-Based Migration
The host copies data from old to new storage using OS tools (like dd or robocopy). This requires:
- Taking applications offline during copy
- Or using application-level replication
- Host resources (CPU, memory) for copying
- Understanding of file systems and applications
This approach is simple and doesn’t require special infrastructure, but it usually requires downtime and doesn’t scale well to many hosts.
Array-Based Migration
Modern arrays can import foreign LUNs and migrate data internally. The array handles the migration transparently to hosts.
Advantages:
- Efficient use of array resources
- Transparent to hosts
- Often non-disruptive
Disadvantages:
- Vendor lock-in (usually works only with same-vendor arrays)
- Requires specific array support
- May require expensive licensing
Network-Based Migration
This is where FC-Redirect comes in. The network fabric virtualizes storage and manages migration:
- Works with any storage array
- Completely transparent to hosts
- Managed centrally
- Requires capable switches
Network-based migration provides maximum flexibility and is especially valuable in heterogeneous environments.
FC-Redirect Migration Architecture
Let me explain how FC-Redirect handles migration:
Initial State: Hosts access LUNs on the source array. We configure FC-Redirect to create virtual LUNs that initially map directly to the source array. Hosts are zoned to see the virtual LUNs instead of physical LUNs.
Migration Start: We add the destination array and create target LUNs. We initiate a migration job that begins copying data from source to destination in the background.
Active-Active Phase: While copying proceeds, I/O continues. Reads are served from the source array. Writes go to both source and destination arrays (or just destination, depending on migration progress). This ensures consistency.
Verification: After copying completes, we verify the destination data matches the source. Any differences trigger re-copying of affected blocks.
Cutover: Once verification passes, we atomically switch the virtual LUN to point to the destination array. From the host’s perspective, nothing changes—the virtual LUN still has the same WWPN and LUN number.
Cleanup: We can now remove the source array from the configuration and reclaim its storage.
This entire process happens while applications continue running. The host never knows the underlying storage changed.
Data Consistency
Maintaining consistency during migration is critical. Imagine this scenario:
- Block A contains data “X”
- Migration copies block A (containing “X”) to destination
- Application writes “Y” to block A on source
- Migration completes and switches to destination
- Block A on destination still contains “X”, but it should contain “Y”
To prevent this, we must ensure writes during migration are handled correctly:
Write-Through: Writes go to both source and destination during migration. This ensures consistency but doubles write I/O.
Change Tracking: Track which blocks are written during migration and re-copy them before completion. This is more efficient but more complex.
Snapshots: Take a snapshot of source before migration, copy from snapshot. Writes go to normal volume. Reconcile at the end.
FC-Redirect uses sophisticated change tracking to minimize overhead while ensuring consistency.
Migration Performance
Migration performance matters because slow migrations take longer, increasing risk and resource consumption.
Factors affecting migration performance:
Available Bandwidth: Migration uses fabric bandwidth. Ensure adequate bandwidth is available without impacting production I/O.
Array Performance: Both source and destination arrays must handle migration I/O along with production I/O.
Optimization: Sequential reads from source are more efficient than random reads. Migrating in LUN order optimizes source array access patterns.
Throttling: Migration should be throttled to avoid impacting production I/O. We provide controls to limit migration bandwidth.
In practice, we can often migrate at hundreds of MB/s per LUN while keeping production impact minimal.
Scheduling and Throttling
Migrations should be scheduled carefully:
Off-Peak Hours: Run intensive migration during low-usage periods to minimize impact.
Throttle Rate: Limit migration bandwidth to a percentage of available capacity.
Adaptive Throttling: Monitor production I/O and automatically reduce migration rate when production load increases.
Pause and Resume: Ability to pause migration during critical periods and resume later.
Good migration tools provide fine-grained control over these parameters.
Verification and Testing
Before cutting over to new storage, verify everything is correct:
Data Verification: Compare source and destination block-by-block. This is time-consuming but critical for important data.
Checksum Comparison: Compare checksums rather than full data. Faster but slightly less certain.
Test Cutover: For critical systems, perform a test cutover to a non-production host first to verify the destination storage works correctly.
Backup: Always backup before migration. Despite best efforts, things can go wrong.
Never skip verification for production data. The cost of data corruption far exceeds the time spent verifying.
Rollback Planning
Migrations can fail. You need a rollback plan:
Keep Source Online: Don’t decommission source storage until you’re certain the migration succeeded and the destination is stable.
Quick Rollback: Ensure you can quickly switch back to source if problems arise with the destination.
Documentation: Document exactly how to rollback. In a crisis, you don’t want to be figuring this out.
Testing: Test the rollback procedure before the migration to ensure it works.
With FC-Redirect, rolling back is straightforward—we simply switch the virtual LUN back to the source array. But testing this before production cutover is essential.
Multi-Host Considerations
Most enterprise LUNs are accessed by multiple hosts. Migration gets more complex:
Clustered File Systems: If hosts share a clustered file system (like VMFS or GFS), all hosts must see the migration happen atomically.
Fencing: Some clusters use SCSI reservations for fencing. Ensure these survive migration.
Multipathing: Hosts using multipathing need to discover paths to the new storage and fail over correctly.
Coordination: In some cases, cluster quorum or coordination protocols might need attention during migration.
Testing with all hosts is essential before production migration.
VMware-Specific Considerations
VMware environments have specific considerations:
Storage vMotion: VMware’s built-in migration tool works well but requires downtime for the VM being migrated (though only that VM, not the whole host).
Array Integration: VMware’s VAAI (vStorage APIs for Array Integration) can accelerate migrations by leveraging array-based copy.
VMFS Resignaturing: When presenting a copied VMFS volume, VMware requires resignaturing to avoid UUID conflicts.
Snapshots: VMs with snapshots are more complex to migrate. Consider consolidating snapshots first.
Many VMware shops use a combination of Storage vMotion for VM-level migrations and array or network-based migration for moving entire datastores.
Windows Cluster Considerations
Windows clusters present challenges:
Cluster Validation: After migration, re-run cluster validation to ensure everything still works.
Quorum Configuration: Ensure quorum witnesses (disk or file share) survive migration.
Cluster Shared Volumes (CSV): CSV has specific requirements for storage characteristics. Verify new storage meets them.
Service Interruption: Some cluster migrations may require brief service interruption to update configurations.
Understanding Windows clustering deeply is essential for successful migration in Windows environments.
Migration Tools and Automation
Manual migration is error-prone. Good tools provide:
Discovery: Automatically discover storage topology—which hosts see which LUNs on which arrays.
Planning: Model the migration, verify capacity and performance adequacy.
Orchestration: Automate the migration steps with appropriate sequencing and error handling.
Monitoring: Real-time visibility into migration progress and any issues.
Reporting: Documentation of what was migrated and verification results.
For large migrations, automation is essential. Migrating hundreds of LUNs manually is risky and tedious.
Lessons Learned
Having worked on many migrations, here are key lessons:
Start Small: Migrate a small, non-critical system first to validate the process.
Test Everything: Test discovery, migration, verification, cutover, and rollback before production.
Expect the Unexpected: Something will always surprise you. Leave buffer in your schedule.
Communication: Keep stakeholders informed. Surprises during migration create stress.
Documentation: Document everything—configuration, procedures, issues, resolutions.
Verify, Verify, Verify: Data corruption is catastrophic. Always verify.
Don’t Rush: Hasty migrations lead to mistakes. Better to take extra time than to corrupt data.
Advanced Topics
For very large or complex migrations:
Phased Migration: Migrate in phases rather than all at once. This limits risk and allows learning between phases.
Hybrid State: Run some workloads on old storage and some on new during transition period.
Migration Waves: Group LUNs into waves based on application, criticality, or other factors. Migrate each wave separately.
Performance Testing: After migration, verify performance meets expectations. Sometimes new storage behaves differently than expected.
The Future of Migration
Migration technologies continue evolving:
Greater Automation: More intelligent tools that require less manual intervention.
Continuous Migration: Rather than discrete migration projects, continuous data movement based on policies.
Cloud Integration: Tools that migrate between on-premises and cloud storage seamlessly.
Predictive Analytics: Using analytics to optimize migration timing and sequencing.
These advances will make migrations less disruptive and more routine.
Conclusion
Storage migration is a critical skill for storage administrators. Technology refresh cycles ensure you’ll be migrating regularly throughout your career.
The key to successful migration is thorough planning, comprehensive testing, good tools, and systematic execution. Never rush a migration, always verify data integrity, and always have a rollback plan.
FC-Redirect makes network-based migration practical for FC environments. The ability to migrate between heterogeneous arrays without host involvement is powerful. Combined with proper methodology, it enables non-disruptive migrations that would be difficult or impossible otherwise.
Whether using host-based, array-based, or network-based migration, understanding the principles and best practices ensures your migrations are successful and stress-free. Well, maybe not stress-free, but at least successful.