How do I use OverlayFS?

1

Edit: Since writing this answer, some things have changed in overlayfs, namely the addition of a required parameter workdir, see totti's answer below for a detailed description of this new parameter.

I finally managed to find it. I found references to it in the kernel source, but for some reason it doesn't appear in the git tree on kernel.org. But! If you pull the Ubuntu kernel source like this: apt-get source linux-image-3.0.0-16-generic you can find it in linux-3.0.0/Documentation/overlayfs.txt. It is also available in the linux-doc package in /usr/share/doc/linux-doc/filesystems/overlayfs.txt.gz.

As the actual help documentation is more of a "how it works" instead of a "how to mount with it," here's a brief rundown (there is one example in the kernel documentation):

mount -t overlayfs -o [mount options] overlayfs [mountpoint for merged system]

Where [mount options] can be:

lowerdir=somedir: lowerdir is the directory you're going to lay your new...
0 0
2

This answer and email message indicate that something called "OverlayFS" is available in Ubuntu 11.10 and will forcefully replace aufs in Ubuntu 12.04.

Edit: Since writing this answer, some things have changed in overlayfs, namely the addition of a required parameter workdir, see totti's answer below for a detailed description of this new parameter.

I finally managed to find it. I found references to it in the kernel source, but for some reason it doesn't appear in the git tree on kernel.org. But! If you pull the Ubuntu kernel source like this: apt-get source linux-image-3.0.0-16-generic you can find it in linux-3.0.0/Documentation/overlayfs.txt. It is also available in the linux-doc package in /usr/share/doc/linux-doc/filesystems/overlayfs.txt.gz.

As the actual help documentation is more of a "how it works" instead of a "how to mount with it," here's a brief rundown (there is one example in the kernel documentation):

mount -t overlayfs -o [mount...

0 0
3

According this document it should be possible to use multiple lower layers with overlayfs, but I can't get it to work on kernel 3.18.13. Am I using it wrong, or do I need a later kernel, and if so which one?

this works:

>>>mkdir lower1 lower2 upper workdir merged >>>echo hello > lower1/myfile >>>sudo mount -t overlay overlay -olowerdir=lower1,upperdir=upper,workdir=workdir/ merged >>>cat merged/myfile hello

example very similar to document:

>>>sudo mount -t overlay overlay -olowerdir=lower1:lower2 merged mount: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. >>>dmesg | tail -n1 [42131.327203] overlayfs: missing upperdir or lowerdir or workdir

ok, lets add them:

>>>sudo mount -t overlay overlay -olowerdir=lower1:lower2,upperdir=upper,workdir=workdir/ merged mount: wrong fs type, bad option, bad superblock on overlay, ...
0 0
4
...
0 0
5

I am trying to aggregate two mount points in to a single directory so that it will combine the files from both in to a single mountpoint. I am doing some tests with overlayfs and have been able to create the merged mountpoint, but the issue comes when I delete files from this merged directory.

I understand that overlayfs has an upper and a lower filesystem and that if you delete something from the lower that it creates a "whiteout" on the upper filesystem that basically just hides the file from the lower from "shining through" to the merged layer. The file still occupies space on the lower filesystem; it's just not presented as available.

My question is: how do I delete a file from both the upper and lower filesystems and reclaim the space used by the deleted file, and not just hide it? I can manually delete the file from the merged directory and then from the lower directory (is this a bad thing to do?) but df -ha still shows the merged directory as taking up...

0 0
6

There has been a similar question with an answer pointing towards Overlay being built into the 3.19 Linux Kernel that will support multiple lower layers. The link an answer provides (and other pages I’ve stumbled across) doesn’t supply any information about how to do this in Linux 3.19 and only states that it is possible.

I am currently running on 3.19 an having a problem trying to get multiple lower layers to overlay.

From what I’ve read to mount an overlay with multiple lowerdirs looks like this

mount -t overlay overlay -o lowerdir=/firstlower:/secondlower,upperdir=/upper/root,workdir=/upper/workdir /merged

where the two lower layers are colon-separated. I’ve not been able to find a reliable enough source to confirm this information though so I’m hoping this post will help.

Using the above code renders an error:

overlayfs: failed to resolve '/firstlower:/secondlower': -2 mount: mounting overlay on /merged failed: Invalid argument

Does...

0 0
7

Posted by:

Thom Denholm

Union file systems are a creative solution to allow a virtual merge of multiple folders, while keeping their actual contents separate. The Overlay file system (OverlayFS) is one example of these, though it is more of a mounting mechanism than a file system.

Brought into the Linux kernel mainline with version 3.18, OverlayFS allows you to overlay the contents (both files and directories) of one directory onto another. The source directories can be on different volumes and can even be different file systems, which creates an interesting mechanism for allowing temporary modification of read-only files and folders.

The simplest case (image below) involves two directories, each containing files and folders. We can think of them as “upper” and “lower,” with the rest of Linux and applications positioned above that. The “lower” directory is read-only. File access through the OverlayFS retrieves data from the “upper” directory...

0 0
8

This community looks very active, and I hope I can catch hold of someone who can guide me with JFFS2 filesystem, and hw overlayfs can be used with it.

I've read the wiki, and I seem to be able create a filesystem that is mountable.
Currently, I have an empty directory with 1 file, that I would like to use as home.

jlumme@simppa:~/projects/jffs2_home$ ls -la total 20 drwxrwxr-x 4 jlumme jlumme 4096 Apr 21 16:21 . drwxrwxr-x 6 jlumme jlumme 4096 Apr 21 16:21 .. drwxrwxr-x 2 jlumme jlumme 4096 Apr 21 13:45 default drwxrwxr-x 2 jlumme jlumme 4096 Apr 21 13:45 ftp -rw-rw-r-- 1 jlumme jlumme 24 Apr 21 15:34 test.txt

The flash chip I'm using is SST25VF064C, and from it's datasheet I can see that it has erase block size of 64KB, so I use mkfs.jffs2 to create a filesystem image:

mkfs.jffs2 -r jffs2_home/ -e 64 -o home.jffs2 $ ls -la -rw-r--r-- 1 jlumme jlumme 496 Apr 21 15:42 home.jffs2

My flash layout looks like this:

uboot/ kernel/ dtb/...
0 0
9

OverlayFS provides a great way to merge directories or filesystems such that one of the filesystems (called the "lower" one) never gets written to, but all changes are made to the "upper" one. This is great for systems that rely on having read-only data, but the view needs to be editable, such as Live CD's and Docker containers/images (image is read only). This also allows you to quickly add some storage to an existing filesystem that is running out of space, without having to alter any structures. It could also be a useful component of a backup/snapshot system.

[Image courtesy of Docker Docs]

To get a much better understanding of what overlayFS is and how it works, please read this article provided by Docker. The rest of this tutorial will focus on using OverlayFS.

Simplest Case - Directories Only

The command below creates directories for the lower filesystem, lower filesystem and the merged result which goes into "overlay" (but you may wish to rename...

0 0
10

I'm having the following problem with an overlayfs: I can mount an overlayfs fine using the following line in my /etc/fstab:

none /home/student overlayfs lowerdir=/home/student,upperdir=/home/.student_rw 0 0

This works great and the mount command displays the overlayfs correctly:

pc:~$ mount -l -t overlayfs none on /home/student type overlayfs (rw,lowerdir=/home/student,upperdir=/home/.student_rw)

So now when a student logs out of the machine I delete the contents of /home/.student_rw using a script – which also works fine.

The problem is that the student logging in should find a clean home directory with the settings preconfigured in /home/student as the overlayfs does not touch the lowerdir folder. However overlayfs "remembers" the deleted files and does not show them, even if they exist in the lowerdir. After a reboot this approach works fine. I want that to work on logout and login though. How can I fix this problem? For me a possible solution seems to be...

0 0
11

OverlayFS is a modern union filesystem that is similar to AUFS, but faster and with a simpler implementation. Docker provides two storage drivers for OverlayFS: the original overlay, and the newer and more stable overlay2.

This topic refers to the Linux kernel driver as OverlayFS and to the Docker storage driver as overlay or overlay2.

Note: If you use OverlayFS, use the overlay2 driver rather than the overlay driver, because it is more efficient in terms of inode utilization. To use the new driver, you need version 4.0 or higher of the Linux kernel, unless you are a Docker EE user on RHEL or CentOS, in which case you need version 3.10.0-693 or higher of the kernel and to follow some extra steps.

For more information about differences between overlay vs overlay2, refer to Select a storage driver.

Prerequisites

OverlayFS is supported if you meet the following prerequisites:

The overlay2 driver is supported for Docker EE and...

0 0
12

There has been a similar question with an answer pointing towards Overlay being built into the 3.19 Linux Kernel that will support multiple lower layers. The link an answer provides (and other pages I've stumbled across) doesn't supply any information about how to do this in Linux 3.19 and only states that it is possible.

I am currently running on 3.19 an having a problem trying to get multiple lower layers to overlay.

From what I've read to mount an overlay with multiple lowerdirs looks like this

mount -t overlay overlay -o lowerdir=/firstlower:/secondlower,upperdir=/upper/root,workdir=/upper/workdir /merged

where the two lower layers are colon-separated. I've not been able to find a reliable enough source to confirm this information though so I'm hoping this post will help.

Using the above code renders an error:

overlayfs: failed to resolve '/firstlower:/secondlower': -2 mount: mounting overlay on /merged failed: Invalid argument

Does...

0 0
13

One should be able to combine multiple lower directories in an OverlayFS mount by using a colon to separate them according to this document.For example if I setup with:

mkdir /tmp/data
mkdir -p /tmp/storage/1448918558
mkdir /tmp/storage/1448918559
mkdir /tmp/storage/1448918560
mkdir /tmp/workdir

And then run

sudo mount -t overlay overlay -o \
lowerdir=/tmp/storage/1448918559:/tmp/storage/1448918558,\
upperdir=/tmp/storage/1448918560,\
workdir=/tmp/workdir \
/tmp/data

I get the error message:

mount: wrong fs type, bad option, bad superblock on overlay,
missing codepage or helper program, or other error
(for several filesystems...

0 0
14

Top filesystem frequently asked interview questions

How do I avoid the "S to Skip" message on boot?

After upgrading my laptop from karmic to lucid, my fat32 partition won't mount automatically. I get the message:

The disk drive for /osshare is not ready yet or not present Continue to wait; or Press S to skip mounting or M for manual recovery

Funny thing is, if I skip, then /osshare/ is mounted once I log in.

I've a similar setup on my desktop, and it works fine. Fstab on desktop:

UUID=4663-6853 /osshare vfat utf8,umask=007,gid=46 0 1

/etc/fstab on laptop:

UUID=1234-5678 /osshare vfat utf8,auto,rw,user 0 0
Source: (StackOverflow)

How is 'rm -rf /' able to delete all files in the system?

I haven't tried this command on Ubuntu (for obvious reasons) so I am not sure if Ubuntu will allow its execution. But it's famous for deleting everything. Just out of curiosity, what happens when the kernel and /bin are deleted? How does...

0 0
15

Written by: Neil Brown Please see MAINTAINERS file for where to send questions. Overlay Filesystem ================== This document describes a prototype for a new approach to providing overlay-filesystem functionality in Linux (sometimes referred to as union-filesystems). An overlay-filesystem tries to present a filesystem which is the result over overlaying one filesystem on top of the other. The result will inevitably fail to look exactly like a normal filesystem for various technical reasons. The expectation is that many use cases will be able to ignore these differences. This approach is 'hybrid' because the objects that appear in the filesystem do not all appear to belong to that filesystem. In many cases an object accessed in the union will be indistinguishable from accessing the corresponding object from the original filesystem. This is most obvious from the 'st_dev' field returned by stat(2). While directories will report an st_dev from the overlay-filesystem,...

0 0
16

I'm using overlayFS to create a sort of simple experimental sandboxing. I think I understand how it works:

lower layer should be always mounted R/O upper layer is where all changes happen work layer is for transactional reasons target is where all merged lower + upper go

As example, I have a physical disk mounted on /disk2. Then I want to overlay to preserve the contents, so I can do something like:

mount("overlay", "/disk2", "overlay", MS_MGC_VAL, "lowerdir=/disk2,upperdir=./upper,workdir=./work");

This works just fine: when my new process tries to amend content under /disk2, then it will only amend in reality under upper: this is indeed working as intended. For example, I can also see the 0, 0 files to show that something is deleted in the upper but not in the lower.

Unfortunately looks like I can't somehow overlay on root /: if I execute the following code:

mount("overlay", "/", "overlay", MS_MGC_VAL, "lowerdir=/disk2,upperdir=./upper,workdir=./work")...
0 0
17

I'm running Ubuntu 14.04 with the 3.19 kernel which should have OpenFS supported (since kernel 3.18). I've been reading about OverlayFS but am confused as to how to actually implement using it. It looks like it could be a really useful alternative to LVM in enabling changes to be written somewhere else whilst taking a backup of the original filesystem that is being overlayed?

Can someone give me the example mount commands to use (if possible) for the following cases:

With a raw disk image (created using dd if=/dev/zero of=$HOME/filestystem1.img bs=4096 count=1024000) to act as the overlaying filesystem, mount it over the top of my host's filesystem at $HOME/mount which should already contain some random files such as file1.txt and file2.txt. Once mounted I believe that I should still be able to see my already existing files and any files I now create will actually be stored inside the filesystem1.img file that I could possibly move to another system?

Create...

0 0
18

OverlayFS has a workdir option, beside two other directories lowerdir and upperdir, which needs to be an empty directory.

Unfortunately the kernel documentation of overlayfs does not talk much about the purpose of this option.

The "workdir" needs to be an empty directory on the same filesystem as upperdir.

For readonly overlays the workdir might be ommittet among the upperdir. This give me the clue that it has to do with writing the merged files.

Please explain what's happening in the workdir when files are written or changed in the merged directory. Why is the writable upperdir not enough?

The workdir option is required, and used to prepare files before they are switched to the overlay destination in an atomic action (the workdir needs to be on the same filesystem as the upperdir).

Source: http://windsock.io/the-overlay-filesystem/

I would hazard a guess that "the overlay destination" means upperdir.

So... certain files...

0 0
19

Rather than take an 'All or nothing' approach to containers (eg FreeBSD/Solaris/OpenVZ), native Linux Containers support allows you to unshare Specific resources from the host. These can be mixed and matched in various ways to produce interesting combinations for things such as testing network setups, preventing information leakage (eg for shared hosting webservers) or testing out OS builds (eg from debootstrap). It can even be used to provide a more complete fakeroot replacement

UTS: Allows a different hostname for each container PID: Hides processes outside the namespace from processes in the namespace, calling shutdown() in a PID namespace will perform a 'shutdown' of processes only in that namespace (in Linux > 3.8) MOUNT: Allows a group of processes to mount and unmount filesystems and not have these changes visible outside the namespace UID: Allows you to give processes root inside the namespace but have this mapped to a...
0 0