How to choose the right data storage?

We keep photos, videos, and documents on disks, store company data or can run virtual machines. However, when the need for cooperation arises, such a distributed form will not be useful. To ensure 24/7 access to data, file exchange, an appropriate level of control and efficient communication, a centralized solution is necessary. Regardless of whether it is a matrix, NAS server or external hosting, a business needs a place to store files. A safe and reliable place that will also meet all expectations. So how do you choose the right data storage system for your company?

Know your needs

There is no universal solution that will work well for storing data in a small, medium or large company. The nature of the business plays a huge role here. A small law firm will have completely different needs than a 3-person team producing video ads. A large consulting company that works mainly on spreadsheets or text files will have different technology than a factory that collects data from thousands of sensors.

Each data storage system also has its advantages and disadvantages. For example, NAS solutions are cheap and very capacious, but in the event of a failure, there is a risk of data loss. Traditional arrays are well scalable, but due to limited performance they will not work in the case of a large application load. The most advanced are AllFlash systems, but due to high initial costs they are not the first choice for all organizations. An alternative is hosting with external partners, but despite easy scalability, many organizations have concerns about security and privacy. And yet we all want our data to be stored “in good hands”.

With so many options available, instead of acting on your own, it is worth relying on experts who, in addition to knowing market trends, have years of experience in selecting offers and implementing data storage solutions and will help you choose devices that will take into account the future development of your company.

How to choose the right data storage?

In an effort to match file storage technology to the needs of customers and users, our engineers ask a series of questions that will help tailor the solution. Some issues can be thought through in advance and answers prepared, thus speeding up the entire process.

What is the company’s business profile?

It very often determines the technology used, the type of files stored, and the future needs of the organization. If these are tasks that require fast data flow, we look for arrays with a high IOPS parameter (the number of input/output operations per second). If, on the other hand, storage space and/or price are more important than performance, we go for platters with traditional disks or simply NAS servers.

Are there any nuances specific to the industry? There are plenty of them.

Example 1: We mainly store documents

Accounting and trading companies will store the document flow database, excel files, word files, etc. on the matrix. The database is sensitive to the read/write speed (slow data storage significantly affects the comfort of using it), while excel/word files are not necessarily so. Hence, for a small accounting office, you can choose a matrix with traditional disks to store data, but for a larger one, where many accountants work at the same time, with mixed types of disks: some Flash (intended for databases) and some traditional HDD disks (for word/excel files).

Example 2: We work on heavy CAD files

The architectural office will store large, “heavy” CAD files, a slow storage system will significantly affect the comfort of working with such files. Recommended mixed array or AllFlash with deduplication functions (this is rather an industry standard, but it is worth mentioning that this functionality is in the array).

Example 3: We work on large files in real time

A company dealing with graphics and video processing can approach the problem in two ways.

• if it needs space for storing files, then a cheap array based on HDD disks will suffice, performance is not needed there, space is more important,
• if it wants to work on files stored on the array, then the absolute minimum will be a mixed array, with an indication of allflash, because ordinary platter disks will not cope with the task of efficient video editing in real time. In such a business profile, deduplication is also recommended.

Example 4: We store recordings and use OCR

Monitoring in the security department – depending on the needs, if the camera recordings are to be stored only for occasional viewing, the cheapest array based on regular disks will cope with this task. However, if, in addition to storage space, real-time tags are applied to the recordings in the form of identifying subtitles (OCR), people, objects, etc., then it is worth ensuring that it is a more efficient array, based at least on mixed disks, with an active deduplication mechanism.

How much data storage do you need for your data today and in the future?

How much space do you currently use, what is the average growth rate, and what will it look like in the future? As a client choosing a data storage, you need to know how much space your data uses and how quickly it grows. In accounting, the growth is small (accounting may produce thousands of Word/Excel files annually, but in practice it is tens of gigabytes at most, while a video editing studio will record terabytes of recordings in that time). In the other cases mentioned above, the stored data (or more precisely, its volume) can grow much faster.

Specific file formats and database environment

The same design file, video or photo in different formats can weigh even several times more, so it is important to specify them. In the case of a large number of users using these files or very large files, we will need a faster and more efficient array.

This will also be important especially in database environments, where greater use means greater customer needs. Additionally, connections can be sensitive to power failures and instability to the point of causing database failure. When selecting an array for such applications, we must be sure that it will be redundant and equipped with a well-functioning cache.

What servers do you have in your infrastructure?

Not all devices get along perfectly, so we take into account the existing elements of the company’s infrastructure and try to match solutions to them so that they work efficiently. Some of them may require additional switches or additional infrastructure with a specific specification.

Most often, we examine the types of interfaces: we can choose between connections between the NAS server/array and servers via Ethernet, iSCSI, FC and SAS. Each of these connections has its advantages, disadvantages and “environmental requirements” (e.g. FC quite often, although not always, requires an FC switch; for iSCSI, enterprise switches are useful, capable of transporting significant amounts of data via Ethernet, etc.).

What’s your budget for a data storage?

We always try to choose devices taking into account the ratio of real needs to quality and price. However, it is important to remember that the higher class the devices, the more possibilities and security they have.

Data storage parameters worth paying attention to

1. Encrypted disks – will secure stored data
2. IOPS – the higher the value, the faster the data write and read
3. Thin Provisioning – will allow multiple users to share the array space
4. RAID level or other solution minimizing the effects of failure
5. Built-in system for managing the data storage and monitoring parameters
6. FiberChannel support to ensure greater throughput
7. Deduplication with compression – will allow you to reduce the amount of data stored on the array

Better twice too much than once too little

Once we choose a method of storing files, it is worth making sure not to lose them. Good practices and security measures should be implemented from the very beginning.

Emergency plan

Along with the implementation of new solutions, it is good to create backup procedures and an emergency plan in the event of a failure or hacker attack. Practice shows that many organizations do not take threats seriously and believe that they are not affected. However, the effects of negligence can be catastrophic and in extreme cases lead to the collapse of the business.

Testing, testing, testing

Just creating a backup copy is not everything. An important stage of every procedure is testing. This is the moment when it is checked whether the backup copy is working properly and whether it is possible to recover it quickly.

Diversification and multiplication

Each copy is safe to the extent that it has copies. Since disasters usually come in pairs, it is good to protect yourself against every eventuality and, if possible, create more than one backup copy. In ideal conditions, we can diversify not only devices, but also geographical location, e.g. in the event of a fire.

Better safe than sorry

The higher the class of the device, the better the security it has. Dual components are becoming increasingly standard for many solutions, but they are still worth considering. In the event of a power supply failure, the other will still work. If one of the drives fails, the stored data will be duplicated on its spare module. In this eventuality, the “hot swap” function may be important, as it allows for the replacement of a component without having to interrupt work. If we are thinking about the critical infrastructure of a company, it is not worth saving on it.

Find the data storage for your organization

At Grandmetric, we have years of experience in supporting companies in choosing a file storage method. Practice and knowledge confirmed by certificates allow us to match solutions to the needs of our clients. We know the potential threats and challenges, which is why we always include them in our strategy, regardless of the size of the business.

If you want to learn more or have doubts about which solution will be best for you – talk to our engineers!