Skip to main content
Pure1 Support Portal

RapidFile Toolkit for FlashBlade (PureTools)

RapidFile Toolkit 1.0 Beta (formerly, PureTools) is a set of supercharged Linux command-line utilities for finding, managing, and copying millions of files stored on FlashBlade. RapidFile provides fast client-side alternatives for common Linux commands -- ls, dufind, chown, chmod, rm, and cp -- that have been optimized for the high level of concurrency supported by FlashBlade NFS. RapidFile Toolkit includes all of the tools previously in PureTools and adds the blazing fast pcp file copy tool for FlashBlade.

RapidFile Toolkit 1.0 Beta currently supports searching, copying, and reporting disk usage, with the following commands: pls, pdu, pfind, pchown, pchmod, prm, and pcp.

RapidFile Toolkit is currently a beta release and not should be used on production data. RapidFile Toolkit 1.0 Beta has NOT been validated with non-Pure NFS servers.

Downloading RapidFile Toolkit 1.0 Beta

Pure Storage customers and partners can download the RapidFile Toolkit from the following page (login required):
http://support.purestorage.com/FlashBlade/Purity_FB/Purity_FB_Release/02_Purity_FB_Release_Information/PureTools_for_FlashBlade

Install Filename Release Date Version MD5 Features & Enhancements Fixes
rapidfile-1.0.0-beta.3.tar 21 Oct 2019 1.0.0-beta.3+c165774 2cbca2a0c4a5da1b9d1eace4b5421ea4 - Cleaner verbose output

- If a user deletes files from the source directory while pcp is copying them, pcp now prints "Stale file handle" to stderr and continues rather than halting

rapidfile-1.0.0-beta.2.tar 10 Oct 2019 1.0.0-beta.2+70e8b1e e04b8a74e2186cc1331b846fa4b6eb0c

- Added tarball for installing to custom location

- Fixed pcp out of memory error for large directory structures with large file counts per directory

- Fixed pls not handling trailing slash at mount point root

rapidfile-1.0.0-beta.1.tar 1 Oct 2019 1.0.0-beta.1+d624714 6e4908a61fc0f490d0c06695a938593b

- New blazing fast pcp file copy utility for FlashBlade

- Richer pls --json output

- Fixed pdu when used at mount point root

- Fixed pfind unknown arguments error

Release Notifications

To get notified by email whenever a new release is available:

  1. Go to "RapidFile Toolkit Release Announcements" at: https://community.purestorage.com/discussion/10236/rapidfile-toolkit-release-announcements
  2. Click the star icon to the right of "RapidFile Toolkit Release Announcements" to bookmark the discussion thread. (You may need to log in. If it's still not visible, please let us know by clicking the Leave Feedback link below.)
  3. Click the gear icon in the top right, select Preferences, and confirm that "Email" is checked for "Notify me when people comment on my bookmarked discussions."

Installing RapidFile Toolkit 1.0 Beta

  1. Download the rapidfile-1.0.0-beta.3.tar file attached to this page to your Linux system.
  2. Uncompress the downloaded tar file:
    tar -xvf rapidfile-1.0.0-beta.3.tar
  3. Install using the appropriate install package (deb or rpm):
    1. On Debian-based Linux distributions such as Ubuntu that use the "deb" file format:
      sudo dpkg -i rapidfile-1.0.0-beta.3/rapidfile-1.0.0-beta.3-Linux.deb
    2. On RHEL, CentOS, SUSE, Fedora, or other Linux distributions that use the "rpm" file format:
      sudo rpm -U rapidfile-1.0.0-beta.3/rapidfile-1.0.0-beta.3-Linux.rpm
    3. Alternatively, you can also untar the RapidFile Toolkit binaries directly into a shared location (such as /usr/local/bin) as follows:
      tar -C /path/to/bin -xzvf rapidfile-1.0.0-beta.3/rapidfile-1.0.0-beta.3-Linux.tgz
      
  4. Verify that you can use pls, pfind, pdu, pchmod, pchown, prm, andpcp, and their man pages on NFS mount points.

Upgrading from PureTools to RapidFile Toolkit 1.0 Beta

  1. Download the rapidfile-1.0.0-beta.3.tar file attached to this page to your Linux system.
  2. Uncompress the downloaded tar file:
    tar -xvf rapidfile-1.0.0-beta.3.tar
  3. Uninstall the puretools package and install the rapidfile package (deb or rpm):
    1. On Debian-based Linux distributions such as Ubuntu that use the "deb" file format:
      dpkg --remove puretools
      sudo dpkg -i rapidfile-1.0.0-beta.3/rapidfile-1.0.0-beta.3-Linux.deb
    2. On RHEL, CentOS, SUSE, Fedora, or other Linux distributions that use the "rpm" file format:
      sudo rpm -e puretools
      sudo rpm -U rapidfile-1.0.0-beta.3/rapidfile-1.0.0-beta.3-Linux.rpm
  4. Verify that you can use pls, pfind, pdu, pchmod, pchown, prm, andpcp, and their man pages.

Examples

  1. Reporting on files
    1. Disk usage on a mount:
      pdu -hs /path/to/mountpoint
    2. Count the number of files on a mount:
      pfind /path/to/mountpoint -type f | wc -l
    3. Count files ending with an extension, for example, .h:
      pfind /path/to/mountpoint -type f -path "*.h" | wc -l
  2. Finding and listing files with attributes
    1. Find a fil​​​​​e:
      pfind /path/to/mountpoint -type f -name FILETOFIND
    2. List all files and attributes (output similar to ls -l):
      pls -R -l /path/to/mountpoint
    3. List all files and attributes (what NFSv3 getattr would return) as json (output can be processed with jq):
      pls -R --json /path/to/mountpoint
  3. Copying files
    1. Copy a directory and its contents (NOTE: the extra dot after the source path is required):
      pcp -r /path/to/mountpoint/source/. /path/to/mountpoint/target
    2. Copy a directory and its contents, and preserve attributes (may require sudo) - owner, group, chmod permissions, and last-modified timestamp (mtime):
      pcp -r -p /path/to/mountpoint/source/. /path/to/mountpoint/target
    3. Copy only if updated - copy a directory and its contents and preserve attributes (may require sudo) unless last-modified timestamp (mtime) and size match:
      pcp -r -p -u /path/to/mountpoint/source/. /path/to/mountpoint/target
  4. Modifying files
    1. Delete a directory and its contents (as an alternative to mv to .fast-remove):
      prm -r /path/to/mountpoint/dir-to-delete
    2. Change ownership of a directory and its contents:
      pchown -R NEWUSER:NEWGROUP /path/to/mountpoint/dir-to-modify
    3. Change permissions of directory and its contents:
      pchmod -R NEWPERMS /path/to/mountpoint/dir-to-modify

License

Distribution and use of PureTools is governed by the Pure Storage EULA for Plugin / Adaptor / Provider / SDK / Management Pack, which is available at: https://www.purestorage.com/legal/productenduserinfo.html

Limited Support Only

At this time, Pure Storage provides only limited support for the RapidFile Toolkit 1.0 Beta. As it is a beta product, we recommend you do NOT use RapidFile Toolkit 1.0 Beta on production data. We appreciate bug reports and will try to answer any questions you have.

Customer Forums

Slack Channel on code.purestorage: #rapidfile-toolkit

Discussion Forums: https://community.purestorage.com/categories/rapidfile-toolkit

Contact Us

If you have any issues or questions related to the RapidFile Toolkit or PureTools, please contact the Pure Storage Support Team, message us via our customer forum or Slack channel, or submit a comment via the Leave Feedback link below.