Summary: Unprecedented amounts and variety of data are generated continuously by organizations, and enterprises, which can be used to improve all aspects of life. Although the volume of stored data doubles every year, storage capacity costs decline only at a rate of less than 1/5 per year. Thus, a key challenge in the current era of big data management is to minimize data storage costs while maintaining data fidelity and efficient retrieval. To minimize storage cost and reduce response time, RINNOCO LTD employs two data management techniques: 1) a big data signature-based compression tool, and 2) a data decay tool using postdiction based on machine learning. We utilized the innovation voucher to validate the viability of our Signature-Based Compression (SIBACO) tool, the first milestone of our overall goal as a startup company. We debugged, tested, and evaluated SIBACO in different environments using big datasets with diverse characteristics in terms of data types and data distributions provided by ALGOLYSIS LTD.

Details:

Paper: Towards a Signature Based Compression Technique for Big Data Storage

The article discusses the challenge of minimizing data storage costs while maintaining data fidelity and efficient retrieval and proposes a new signature-based compression technique called SIBACO that achieves higher compression ratios and improves retrieval time for data-intensive applications.

PDF PPTX