Data Reduction Rate (DRR) Estimator for FlashBlade
Pure Storage Confidential. Pure Storage provides no guarantees or warranties, express or implied, on the accuracy of the estimates provided. Actual capacity and performance may vary from the calculated estimates.
Question
What is the Data Reduction Ratio (DRR) Estimator tool, and how is it used?
Answer
DRR Estimator tool is a simple and easy to use tool for estimating data compression rate in a customer environment. The tool provides estimates of data reduction that customers can expect when storing their data on Pure FlashBlade appliances.
What does the DRR Estimator tool do?
- Processes all accessible files and folders under the given path.
- Performs computation of 3 compression values:
- FlashBlade 1st Generation
- FB//S200
- FB//S500
- Compression estimates are available in an aggregated manner at multiple levels:
- DRR estimate of the entire path contents.
- DRR estimate for each file type.
- DRR estimate for different file size buckets.
- Provides two different compression estimation options:
- Quick Scan (default): Processes a smaller sample of the file in case of large sized files for compression estimation (and assumes uniform compression across the file). In case of high density folders, a smaller sample set of files is processed for compression estimation.
- Full Scan: Processes all files completely for compression estimation.
What are the considerations while executing the tool?
- DRR Estimator tool can only be executed on a Linux based client.
- The tool does not edit any dataset file/folder and only requires read privileges for the mount/folder for its execution. The tool privileges should be able to walk through the dataset directory layout for its execution.
- Tool execution will put load on the file system and host while reading files.
- Client resource utilization levels may vary depending on the underlying dataset. Executing the tool in non-peak hours is recommended.
- Time taken to get the DRR results may vary depending on the dataset.
- Inaccessible files and folders are ignored in the compression estimation results.
Tool Recommendations
- The following Linux client configuration is recommended for executing the tool:
- 4 CPU/vCPU
- 8GB RAM
- The DRR Estimator tool is supported on POSIX Linux Operating Systems like CentOS, Ubuntu, etc.
- Scanning a small dataset is advised to check resource impact and performance before scanning larger datasets.
- Scan a dataset representative of the overall production workload to arrive at informed compression estimations.
License
Distribution and use of the DRR Estimator tool is governed by the Pure Storage EULA for Plugin / Adaptor / Provider / SDK / Management Pack, which is available at https://www.purestorage.com/legal/productenduserinfo.html. Licenses for open source software used by DRR Estimator tool can be found in the downloadable .tgz file.
Downloading DRR Estimator Tool
Pure Storage customers and partners can download the DRR Estimator Toolkit and user guide by signing in and using the link below.
Download URL (Log-in Required)
https://support.purestorage.com/FlashBlade/FlashBlade_Release/FlashBlade_Release_Information/Data_Reduction_Rate_(DRR)_Estimator_for_Flashblade (If link is disabled, sign in to download)
DRR Estimator tool | Release Date | Download Links | Fingerprints |
---|---|---|---|
1.0.1 (GA) | Mar 01, 2023 |
tar archive: drr-estimator-tool-1.0.1-linux-x86_64.tgz |
tar archive sha256: 62726a46fb9023484c0be0a2eb08f7a86f3caaa9173cd961f8eb49a93427e828 |
1.0.0 (GA) | Jan 30, 2023 |
tar archive: drr-estimator-tool-1.0.0-linux-x86_64.tgz |
tar archive sha256: 0b9ad31150d8d52ee3715213318280f494d7423ecd2399d9838a6e5a9af04de3 |
Examples
-
Display the help menu:
./pure_drr_estimator -h Pure Data Reduction Rate Estimator Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html. The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com Usage: pure_drr_estimator [OPTIONS]... DIRECTORY Options: -f, --full Perform Full Scan and compression processing. Default is Quick Scan. NOTE: Full Scan performs the compression on the entire contents of the selected folder. As such, the operation may take time depending on the density and size of the folder. Common Options: -h, --help Show this message -V, --version Show version Report bugs and find online help at:https://support.purestorage.com/Solutions/Linux/Linux_Reference/Data_Reduction_Rate_(DRR)_Estimator_for_Flashblade
-
Perform DRR estimation scan of a sample dataset(default is quick scan):
./pure_drr_estimator /path/to/directory Pure Data Reduction Rate Estimator Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html. The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com Disclaimer: Quick scan DRR results may sometimes be unable to give a correct estimation for certain pre-compressed file types. It is recommended to run a full scan to arrive at accurate estimations Version: v1.0 Scan performed: Quick Scan Timestamp: 1673585697 ( 01/12/2023 23:54:00 EST ) Dataset Details: Total files scanned: 10000 Total folders scanned: 1 Inaccessible/Skipped files: 0 Inaccessible/Skipped folders: 0 Top level Compression Details: Actual Size FB-1st-Gen FB//S200 FB//S500 1.72 GB 11.22 MB (157.35:1) 1.95 MB (907.84:1) 1.95 MB (907.84:1) Compression Details by File Size: Type Count Actual Size FB-1st-Gen FB//S200 FB//S500 <4KB 3000 9.54 MB 165.04 KB (59.17:1) 82.03 KB (119.05:1) 82.03 KB (119.05:1) 4KB-16KB 1000 7.63 MB 80.08 KB (97.56:1) 27.34 KB (285.71:1) 27.34 KB (285.71:1) 16KB-32KB 1000 19.07 MB 156.25 KB (125.00:1) 42.97 KB (454.55:1) 42.97 KB (454.55:1) 32KB-128KB 2000 135.42 MB 923.83 KB (150.11:1) 179.69 KB (771.74:1) 179.69 KB (771.74:1) 128KB-512KB 1000 143.05 MB 940.43 KB (155.76:1) 167.97 KB (872.09:1) 167.97 KB(872.09:1) >512KB 2000 1.42 GB 9.01 MB (161.06:1) 1.46 MB (996.07:1) 1.46 MB (996.07:1) Compression Details by File type: Type Count Actual Size FB-1st-Gen FB//S200 FB//S500 .c 1000 47.68 MB 339.84 KB (143.68:1) 74.22 KB (657.89:1) 74.22 KB (657.89:1) .cpp 1000 143.05 MB 940.43 KB (155.76:1) 167.97 KB (872.09:1) 167.97 KB (872.09:1) .doc 1000 3.81 MB 58.59 KB (66.67:1) 27.34 KB (142.86:1) 27.34 KB (142.86:1) .iso 1000 572.20 MB 3.56 MB (160.90:1) 589.84 KB (993.38:1) 589.84 KB (993.38:1) .ost 1000 1.91 MB 47.85 KB (40.82:1) 27.34 KB (71.43:1) 27.34 KB (71.43:1) .ppt 1000 19.07 MB 156.25 KB (125.00:1) 42.97 KB (454.55:1) 42.97 KB (454.55:1) .pst 1000 87.74 MB 583.98 KB (153.85:1) 105.47 KB (851.85:1) 105.47 KB (851.85:1) .txt 1000 3.81 MB 58.59 KB (66.67:1) 27.34 KB (142.86:1) 27.34 KB (142.86:1) .xlx 1000 7.63 MB 80.08 KB (97.56:1) 27.34 KB (285.71:1) 27.34 KB (285.71:1) .xyz 1000 879.29 MB 5.46 MB (161.16:1) 902.34 KB (997.84:1) 902.34 KB (997.84:1) Pure DRR Estimator took 17 Seconds
-
Perform full DRR estimation scan of a sample dataset:
./pure_drr_estimator -f /path/to/directory Pure Data Reduction Rate Estimator Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html. The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com Version: v1.0 Scan performed: Full Scan Timestamp: 1673585943 ( 01/12/2023 23:59:00 EST ) Dataset Details: Total files scanned: 100000 Total folders scanned: 1 Inaccessible/Skipped files: 0 Inaccessible/Skipped folders: 0 Top level Compression Details: Actual Size FB-1st-Gen FB//S200 FB//S500 17.25 GB 112.25 MB (157.35:1) 19.45 MB (907.84:1) 19.45 MB (907.84:1) Compression Details by File Size: Type Count Actual Size FB-1st-Gen FB//S200 FB//S500 <4KB 30000 95.37 MB 1.61 MB (59.17:1) 820.31 KB (119.05:1) 820.31 KB (119.05:1) 4KB-16KB 10000 76.29 MB 800.78 KB (97.56:1) 273.44 KB (285.71:1) 273.44 KB (285.71:1) 16KB-32KB 10000 190.73 MB 1.53 MB (125.00:1) 429.69 KB (454.55:1) 429.69 KB (454.55:1) 32KB-128KB 20000 1.32 GB 9.02 MB (150.11:1) 1.75 MB (771.74:1) 1.75 MB (771.74:1) 128KB-512KB 10000 1.40 GB 9.18 MB (155.76:1) 1.64 MB (872.09:1) 1.64 MB (872.09:1) >512KB 20000 14.17 GB 90.12 MB (161.06:1) 14.57 MB (996.07:1) 14.57 MB (996.07:1) Compression Details by File type: Type Count Actual Size FB-1st-Gen FB//S200 FB//S500 .c 10000 476.84 MB 3.32 MB (143.68:1) 742.19 KB (657.89:1) 742.19 KB (657.89:1) .cpp 10000 1.40 GB 9.18 MB (155.76:1) 1.64 MB (872.09:1) 1.64 MB (872.09:1) .doc 10000 38.15 MB 585.94 KB (66.67:1) 273.44 KB (142.86:1) 273.44 KB (142.86:1) .iso 10000 5.59 GB 35.56 MB (160.90:1) 5.76 MB (993.38:1) 5.76 MB (993.38:1) .ost 10000 19.07 MB 478.52 KB (40.82:1) 273.44 KB (71.43:1) 273.44 KB (71.43:1) .ppt 10000 190.73 MB 1.53 MB (125.00:1) 429.69 KB (454.55:1) 429.69 KB (454.55:1) .pst 10000 877.38 MB 5.70 MB (153.85:1) 1.03 MB (851.85:1) 1.03 MB (851.85:1) .txt 10000 38.15 MB 585.94 KB (66.67:1) 273.44 KB (142.86:1) 273.44 KB (142.86:1) .xlx 10000 76.29 MB 800.78 KB (97.56:1) 273.44 KB (285.71:1) 273.44 KB (285.71:1) .xyz 10000 8.59 GB 54.56 MB (161.16:1) 8.81 MB (997.84:1) 8.81 MB (997.84:1) Pure DRR Estimator took 3 Minutes 44 Seconds
Additional Information
Support
If you are a registered Pure Storage user and need assistance with DRR Estimator tool, please contact Pure Technical Services at https://support.purestorage.com/Pure_Storage_Technical_Services/Technical_Services_Information/Contact_Us.
Contact Us
If you would like to submit product feedback on DRR Estimator tool, please submit a comment via the Leave Feedback link below.