Performing File Operations on Nerves Hosts

Sometimes, you may want to perform operations on files on a host running Nerves (such as a Raspberry Pi). This could include tasks like placing files from your local machine onto the device or syncing Livebook between your local machine and the device.

In this article, I will briefly introduce how to achieve this using rclone.

What is rclone

Rclone is a tool for managing files on various remote storage platforms. It supports cloud storage services like Dropbox and Amazon S3, as well as protocols such as WebDAV and SFTP.

For installation and initial setup, you can refer to the official documentation or articles like "Trying Rclone instead of sshfs since its development ended / Developer Blog & Events | GMO Developers" (in Japanese).

Case Study

Placing files

If it's a small file, you can just bake it into the firmware image, but that's not the case for large files such as machine learning models (in fact, even on the rpi4, the firmware image size is limited to 128MB). Therefore, you need to transfer the files after Nerves has started.

In this example, we will place a machine learning model to be used on Nerves.

Create a directory to store the model (livebook.nerves.local is the connection information; the same applies hereafter):

rclone mkdir livebook.nerves.local:/root/models

Upload the model file (mobilenet.dets) from your local machine:

rclone moveto mobilenet.dets livebook.nerves.local:/root/models/mobilenet.dets

Check if the file has been placed on the Nerves host:

rclone ls livebook.nerves.local:/root/models
     5464 mobilenet.dets

The file has been placed successfully.

Syncing files

As I wrote in a previous article, "Syncing Livebook running on Nerves between the device and a local machine", if you want to manage Livebook files created on Nerves in a repository, you'll want to sync the files on the device to your local development machine. While the previous article achieved this with an SFTP batch process, it can also be done using the rclone sync command.

Create a directory to sync to on your local machine:

mkdir livebook

Sync the directory where Livebook is located on Nerves:

rclone sync livebook.nerves.local:/root/livebook livebook

Check the destination directory on your local machine:

tree livebook
livebook
├── samples
│   ├── basics
│   │   ├── blink.livemd
│   │   ├── button.livemd
│   │   ├── pwm.livemd
│   │   ├── spi_rgb_leds.livemd
│   │   └── sys_class_leds.livemd
│   ├── bluetooth
│   │   └── ble_device_with_nerves.livemd
│   ├── debug
│   │   └── diagnostics.livemd
│   ├── networking
│   │   ├── configure_wifi.livemd
│   │   ├── delta_firmware_update.livemd
│   │   ├── erlang_distribution.livemd
│   │   ├── firmware_update.livemd
│   │   ├── nerves_key_provisioning.livemd
│   │   ├── penultimate_update.livemd
│   │   ├── vintage_net.livemd
│   │   └── weather.livemd
│   └── sensors
│       └── bmp280.livemd
└── welcome.livemd

6 directories, 17 files

We have successfully synced the Livebook files on the device to the local machine.

Conclusion

In this article, I introduced two cases that you might often encounter when manipulating files on Nerves. Rclone has many other features. Try running rclone --help to check them out.