Skip to main content
Contact Us
Register
Pure Technical Services

Data Reduction Rate (DRR) Estimator for FlashBlade

  1. Last updated
  2. Save as PDF

Currently viewing public documentation. Please login to access the full scope of documentation.

Pure Storage Confidential. Pure Storage provides no guarantees or warranties, express or implied, on the accuracy of the estimates provided. Actual capacity and performance may vary from the calculated estimates.

Question

What is the Data Reduction Ratio (DRR) Estimator tool, and how is it used?

Answer

DRR Estimator tool is a simple and easy to use tool for estimating data compression rate in a customer environment. The tool provides estimates of data reduction that customers can expect when storing their data on Pure FlashBlade appliances.

What does the DRR Estimator tool do?

  1. Processes all accessible files and folders under the given path.
  2. Performs computation of 3 compression values:
    • FlashBlade 1st Generation 
    • FB//S200
    • FB//S500
  3. Compression estimates are available in an aggregated manner at multiple levels:
    • DRR estimate of the entire path contents.
    • DRR estimate for each file type.
    • DRR estimate for different file size buckets.
  4. Provides two different compression estimation options:
    • Quick Scan (default): Processes a smaller sample of the file in case of large sized files for compression estimation (and assumes uniform compression across the file). In case of high density folders, a smaller sample set of files is processed for compression estimation.
    • Full Scan: Processes all files completely for compression estimation.

What are the considerations while executing the tool?

  1. DRR Estimator tool can only be executed on a Linux based client.
  2. The tool does not edit any dataset file/folder and only requires read privileges for the mount/folder for its execution. The tool privileges should be able to walk through the dataset directory layout for its execution.
  3. Tool execution will put load on the file system and host while reading files.
  4. Client resource utilization levels may vary depending on the underlying dataset. Executing the tool in non-peak hours is recommended.
  5. Time taken to get the DRR results may vary depending on the dataset.
  6. Inaccessible files and folders are ignored in the compression estimation results.

Tool Recommendations

  1. The following Linux client configuration is recommended for executing the tool:
    1. 4 CPU/vCPU
    2. 8GB RAM
  2. The DRR Estimator tool is supported on POSIX Linux Operating Systems like CentOS, Ubuntu, etc.
  3. Scanning a small dataset is advised to check resource impact and performance before scanning larger datasets.
  4. Scan a dataset representative of the overall production workload to arrive at informed compression estimations.

License 

Distribution and use of the DRR Estimator tool is governed by the Pure Storage EULA for Plugin / Adaptor / Provider / SDK / Management Pack, which is available at https://www.purestorage.com/legal/productenduserinfo.html. Licenses for open source software used by DRR Estimator tool can be found in the downloadable .tgz file.

Downloading DRR Estimator Tool 

Pure Storage customers and partners can download the DRR Estimator Toolkit and user guide by signing in and using the link below.

Download URL (Log-in Required)

https://support.purestorage.com/FlashBlade/FlashBlade_Release/FlashBlade_Release_Information/Data_Reduction_Rate_(DRR)_Estimator_for_Flashblade (If link is disabled, sign in to download)

DRR Estimator tool     Release Date   Download Links    Fingerprints
1.0.1 (GA) Mar 01, 2023

tar archive:

drr-estimator-tool-1.0.1-linux-x86_64.tgz

 tar archive sha256: 62726a46fb9023484c0be0a2eb08f7a86f3caaa9173cd961f8eb49a93427e828
1.0.0 (GA) Jan 30, 2023

tar archive:

drr-estimator-tool-1.0.0-linux-x86_64.tgz

tar archive sha256: 0b9ad31150d8d52ee3715213318280f494d7423ecd2399d9838a6e5a9af04de3

Examples

  1. Display the help menu:

    ./pure_drr_estimator -h

Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Usage: pure_drr_estimator [OPTIONS]... DIRECTORY

Options:
    -f, --full
            Perform Full Scan and compression processing. Default is Quick Scan.
            NOTE: Full Scan performs the compression on the entire contents of the selected folder. As such, the operation may take time depending on the density and size of the folder.
Common Options:
    -h, --help
            Show this message
    -V, --version
            Show version

Report bugs and find online help at:https://support.purestorage.com/Solutions/Linux/Linux_Reference/Data_Reduction_Rate_(DRR)_Estimator_for_Flashblade

  2. Perform DRR estimation scan of a sample dataset(default is quick scan):

    ./pure_drr_estimator /path/to/directory 

Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Disclaimer: Quick scan DRR results may sometimes be unable to give a correct estimation for certain pre-compressed file types. It is recommended to run a full scan to arrive at accurate estimations

Version: v1.0
Scan performed: Quick Scan
Timestamp: 1673585697 ( 01/12/2023 23:54:00 EST )
Dataset Details:
Total files scanned: 10000
Total folders scanned: 1
Inaccessible/Skipped files: 0
Inaccessible/Skipped folders: 0

Top level Compression Details:
Actual Size         FB-1st-Gen                  FB//S200               FB//S500
1.72 GB             11.22 MB (157.35:1)         1.95 MB (907.84:1)     1.95 MB (907.84:1)

Compression Details by File Size:
Type         Count    Actual Size    FB-1st-Gen                   FB//S200                 FB//S500
<4KB         3000     9.54 MB        165.04 KB (59.17:1)          82.03 KB (119.05:1)      82.03 KB (119.05:1)
4KB-16KB     1000     7.63 MB        80.08 KB (97.56:1)           27.34 KB (285.71:1)      27.34 KB (285.71:1)
16KB-32KB    1000     19.07 MB       156.25 KB (125.00:1)         42.97 KB (454.55:1)      42.97 KB (454.55:1)
32KB-128KB   2000     135.42 MB      923.83 KB (150.11:1)         179.69 KB (771.74:1)     179.69 KB (771.74:1)
128KB-512KB  1000     143.05 MB      940.43 KB (155.76:1)         167.97 KB (872.09:1)     167.97 KB(872.09:1)
>512KB       2000     1.42 GB        9.01 MB (161.06:1)           1.46 MB (996.07:1)       1.46 MB (996.07:1)

Compression Details by File type:
Type     Count    Actual Size        FB-1st-Gen                   FB//S200                 FB//S500
.c       1000     47.68 MB           339.84 KB (143.68:1)         74.22 KB (657.89:1)      74.22 KB (657.89:1)
.cpp     1000     143.05 MB          940.43 KB (155.76:1)         167.97 KB (872.09:1)     167.97 KB (872.09:1)
.doc     1000     3.81 MB            58.59 KB (66.67:1)           27.34 KB (142.86:1)      27.34 KB (142.86:1)
.iso     1000     572.20 MB          3.56 MB (160.90:1)           589.84 KB (993.38:1)     589.84 KB (993.38:1)
.ost     1000     1.91 MB            47.85 KB (40.82:1)           27.34 KB (71.43:1)       27.34 KB (71.43:1)
.ppt     1000     19.07 MB           156.25 KB (125.00:1)         42.97 KB (454.55:1)      42.97 KB (454.55:1)
.pst     1000     87.74 MB           583.98 KB (153.85:1)         105.47 KB (851.85:1)     105.47 KB (851.85:1)
.txt     1000     3.81 MB            58.59 KB (66.67:1)           27.34 KB (142.86:1)      27.34 KB (142.86:1)
.xlx     1000     7.63 MB            80.08 KB (97.56:1)           27.34 KB (285.71:1)      27.34 KB (285.71:1)
.xyz     1000     879.29 MB          5.46 MB (161.16:1)           902.34 KB (997.84:1)     902.34 KB (997.84:1)

Pure DRR Estimator took 17 Seconds

  3. Perform full DRR estimation scan of a sample dataset:

    ./pure_drr_estimator -f /path/to/directory

​​Pure Data Reduction Rate Estimator
Distribution and use of the Pure Data Reduction Rate Estimator v1.0 (including compiled binaries pure_drr_estimator, etc.) is governed by the Pure Storage EULA for Plugin/Adaptor/Provider/SDK/Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html.
The Data Reduction Rate estimator output is purely an estimate based on limited dataset information; actual compression rates may vary. This tool is not to be used for deriving any commitment on compression values. The compression rates are Pure Confidential and are not to be shared with third parties . Please share the below output at drr-estimator-results@purestorage.com

Version: v1.0
Scan performed: Full Scan
Timestamp: 1673585943 ( 01/12/2023 23:59:00 EST )
Dataset Details:
Total files scanned: 100000
Total folders scanned: 1
Inaccessible/Skipped files: 0
Inaccessible/Skipped folders: 0

Top level Compression Details:
Actual Size         FB-1st-Gen                  FB//S200                FB//S500
17.25 GB            112.25 MB (157.35:1)        19.45 MB (907.84:1)     19.45 MB (907.84:1)

Compression Details by File Size:
Type          Count     Actual Size     FB-1st-Gen                 FB//S200                   FB//S500
<4KB          30000     95.37 MB        1.61 MB (59.17:1)          820.31 KB (119.05:1)       820.31 KB (119.05:1)
4KB-16KB      10000     76.29 MB        800.78 KB (97.56:1)        273.44 KB (285.71:1)       273.44 KB (285.71:1)
16KB-32KB     10000     190.73 MB       1.53 MB (125.00:1)         429.69 KB (454.55:1)       429.69 KB (454.55:1)
32KB-128KB    20000     1.32 GB         9.02 MB (150.11:1)         1.75 MB (771.74:1)         1.75 MB (771.74:1)
128KB-512KB   10000     1.40 GB         9.18 MB (155.76:1)         1.64 MB (872.09:1)         1.64 MB (872.09:1)
>512KB        20000     14.17 GB        90.12 MB (161.06:1)        14.57 MB (996.07:1)        14.57 MB (996.07:1)

Compression Details by File type:
Type          Count     Actual Size     FB-1st-Gen                  FB//S200                  FB//S500
.c            10000     476.84 MB       3.32 MB (143.68:1)          742.19 KB (657.89:1)      742.19 KB (657.89:1)
.cpp          10000     1.40 GB         9.18 MB (155.76:1)          1.64 MB (872.09:1)        1.64 MB (872.09:1)
.doc          10000     38.15 MB        585.94 KB (66.67:1)         273.44 KB (142.86:1)      273.44 KB (142.86:1)
.iso          10000     5.59 GB         35.56 MB (160.90:1)         5.76 MB (993.38:1)        5.76 MB (993.38:1)
.ost          10000     19.07 MB        478.52 KB (40.82:1)         273.44 KB (71.43:1)       273.44 KB (71.43:1)
.ppt          10000     190.73 MB       1.53 MB (125.00:1)          429.69 KB (454.55:1)      429.69 KB (454.55:1)
.pst          10000     877.38 MB       5.70 MB (153.85:1)          1.03 MB (851.85:1)        1.03 MB (851.85:1)
.txt          10000     38.15 MB        585.94 KB (66.67:1)         273.44 KB (142.86:1)      273.44 KB (142.86:1)
.xlx          10000     76.29 MB        800.78 KB (97.56:1)         273.44 KB (285.71:1)      273.44 KB (285.71:1)
.xyz          10000     8.59 GB         54.56 MB (161.16:1)         8.81 MB (997.84:1)        8.81 MB (997.84:1)

Pure DRR Estimator took 3 Minutes 44 Seconds

Additional Information

Support

If you are a registered Pure Storage user and need assistance with DRR Estimator tool, please contact Pure Technical Services at https://support.purestorage.com/Pure_Storage_Technical_Services/Technical_Services_Information/Contact_Us.

Contact Us 

If you would like to submit product feedback on DRR Estimator tool, please submit a comment via the Leave Feedback link below.