PyFilesystem – files are files no matter where they live.
When I was learning Linux it was drilled into my head. Everything is a file. What this meant was that all input/output’s (documents, directories, hard-drives, modems, keyboards, printers, network communications, etc. ) were byte steams exposed through the filesystem name space. But what if it’s a windows file?
PyFilesystem to the rescue
If you are writing Python you can obviously use the Python standard library to open files. You can import OS and do some if statements and open files on various systems. Or you can use PyFilesystem and open localfiles, files in zipped archives, on a ftp server, with the same code. This means that you can write your code first and decide where you data is going to live later.
First you’re gonna want to download it: pip install fs.
You can get all of the documentation here. PyFilesystem is a python Filesystem abstraction layer. The idea is that no matter where data lives you can use fs to manipulate it. For example:
with open_fs('.') as fs:
with open_fs('zip://projects.zip') as fs:
with open_fs('ftp://ftp.example.org/projects') as fs:
shows how you could open a directory the same way locally, in a zip archive, and on an ftp server. Using PyFilesystem, if your data moves anywhere, you just update the one line. Also it allows you to not worry about differences in OS’s… This is just a small sample of the types of filesystems PyfileSystem can handle. It can walk all types of file systems including the S3FS Amazon S3 Filesystem…
I plan to make PyFilesystem a standard include in all of my relevant projects; which mostly likely is all my future projets lol 🙂 If you are using it on the reg, let me know how it’s working out for you in the comments below.
Addendum: The author of this package answered a number of questions about it on Reddit. He was very helpful and patient answering questions.