Big Data Requires a Big Data Backup and Recovery Strategy

Big data is arriving from multiple sources at a high velocity, volume and variety. To manage big data, you need a strategy for handing this data that includes a data backup and recovery plan.

big data strategyBig data is being generated by everything around us at all times. Every digital process and social media exchange produces it. Systems, sensors and mobile devices transmit it. The Internet of Things (IoT) is generating the demand for a management plan for Big Data. Data is coming from specialized devices and applications and integrating with business applications. Businesses are integrating these specialized devices and applications with their core business processes for analytics and transactional business processing.

Big data must be incorporated in plans for search, development, governance and analytics. To relieve the pressure that big data is placing on your IT infrastructure, you can host some Big Data and analytics solutions on the cloud. Deciding on the correct mix of cloud storage and in-house storage is critical to a successful implementation that is using Big Data. To achieve economies and efficiencies, you can run certain analytics close to the data, while it is in motion. But for data you elect to store in-house, you can use a defensible disposal strategy that reduces the run rate of storage, legal expense and risk.

A data strategy is never complete without a Data Backup and Recovery strategy. A Big Data implementation presents a need for even more focus on the ability to recover from a catastrophic event quickly. However, if an organization is not staffed or tooled to design and execute a Big Data backup strategy of this level of complexity, there are Data Backup/Recovery Managed Service Providers (DB/R MSP) that provide remote management of the Backup process, along with professional Disaster Backup and Recovery consultation. A company should look to a service provider that specializes in complex backup strategies while leaving the IT staff time to manage the daily tactical and long-term strategic activities.

Develop Backup and Recovery Policy Management to Support GRC

Back up and restore policy management is a critical piece to your GRC (Governance, Risk, Compliance). Policy management encompasses all the rules for where data is stored, how many versions can be stored, and for how long it is stored.

Compliance and Regulation words puzzle piece,business conceptConsider that data is not only to be backed up for protection of a catastrophic event, but, it also needs to be restored for many different purposes and meet varying requirements. This requires  Data Backup strategies must be planned and policies developed to assure all company requirements regarding data retention and recovery are met, such as;

  • Regulatory Requirements
  • Data Breaches
  • Court Orders

Tivoli Storage Manager (TSM) provides backup and restore policy management. Policies in TSM are rules that determine how data is stored and managed. The rules include where the data is initially stored, how many backup versions are kept, how long archive copies are kept, and so on. You can have multiple policies and assign the various policies as needed, or even to specific files. Policies Assign a location in server storage where data is initially stored.

Server storage is divided into storage pools that are groups of storage volumes. Server storage can include hard disk, optical, and tape volumes.

Clients use Tivoli Storage Manager to store data for any of the following purposes:

  •  Backup and restore: The backup process copies data from client systems to server storage to ensure against loss of data that is regularly changed. A policy includes the number of versions and the retention time for those versions. The server retains versions of a file according to this policy, and replaces older versions of the file with newer versions.
  • Archive and retrieve: The archive process copies data from client systems to server storage for long-term storage. The process can optionally delete the archived files from the client systems. The server retains archive copies according to the policy for archive retention time. A client can retrieve an archived copy of a file.
  • Backup set recovery: Backup set recovery is the creation of a complete set of backed-up files for a client. The set of files is called a backup set. A backup set is created on the server from the most recently backed-up files that are already stored in server storage for the client. The policy for the backup set consists of the retention time that you choose when you create the backup set. You can copy a backup set onto compatible portable media, which can then be taken directly to the client for rapid recovery without the use of a network and without having to communicate with the Tivoli Storage Manager server.
  • Migration and recall Migration: is a function of the Tivoli Storage Manager for Space Management (for supported UNIX and Linux systems) and Tivoli Storage Manager HSM for Windows programs. It frees client storage space by copying files from client systems to server storage. On the client, the program replaces the original file with a stub file that points to the migrated file in server storage. Files are recalled to the client systems when needed. The process of migrating and retrieving data through these programs is transparent to users and applications, other than a possible degradation in performance as compared to data stored on locally attached, tier one disk. Policy determines when files are considered for automatic migration. On the UNIX or Linux systems that support the Tivoli Storage Manager for Space Management program, policies determine whether files must be backed up to the server before being migrated. Space management is also integrated with backup. If the file to be backed up is already migrated to server storage, the file is backed up from there.

To learn more about data backup and restore policy management, contact Salvus Data Consultants, Tivoli Storage Management experts.

Enterprises are moving to the Cloud in Progressive Steps

Large enterprise are taking steps toward a total cloud data center.

In the post on GIGAOM’s survey results Survey: What the Enterprise Cloud Needs to Become Business-Critical, they state “Many organizations are now progressing beyond these workloads, putting cloud computing to work in support of business-critical applications and workloads.” The post goes on to say “Sixty-six percent of respondents consider one or more Software-as-a-Service (SaaS) applications to be business-critical today, and a significant number also support critical workloads with public Database-as-a-Service (DBaaS) or Infrastructure-as-a-Service (IaaS) compute and storage offerings.”Complex data

However, adoption of a total cloud environment may continue to be impacted with concerns over security, meeting regulations, network bandwidth, and transition costs.

In response to these issues, an enterprise will first implement non-critical applications. As that goes well, the company will gradually begin to implement mission critical applications until the entire data center is a cloud implementation. Cloud applications will interact with in-house applications for a period of time as all issues of concern are worked out or accepted.

Data backup and recovery can follow a similar progression. The data can alternatively remain within the customer’s network and backed up off site to the customer’s choice backup location. The data backup and recovery processes can be managed in the cloud. The data may or may not be in the cloud with the model of remote data backup administration. When the term “cloud” is mentioned, we visualize an “all or nothing” scenario. We think that the data must be replicated to its backup site. We also imagine that the management of the DR processes and the data together must be in the cloud. But, alternative models are possible.

There are Data Backup/Recovery Managed Service Providers that provide remote management of the Backup process, along with professional Disaster Backup and Recovery consultation. To further discuss the subject of cloud data backup and recovery management, contact Salvus Data Consultants at 903-201-7233