How do I install and use flashcache/bcache to cache HDD to SSD?

1

Prior warning: bcache can not be used for your existing Ubuntu installation. If you need SSD caching for your operating system you should wait for a future or special version of the Ubuntu installer which could set up your SSD cache. See this answer by Oli here: Does bcache require a fresh installation?

For everyone else who wants a bcache drive for non-OS data:

The first thing to do is upgrade your kernel if your using 13.04 or older using the kernel mainline PPA. Instructions for using this can be found here:

https://wiki.ubuntu.com/Kernel/MainlineBuilds

Make sure you select the 3.9 or higher kernel when booting or it won't work and your drive will disappear.

Install bcache-tools, at the moment you will have to install from the git repository:

git clone http://evilpiepirate.org/git/bcache-tools.git

If you manage to package bcache-tools for debian, let us know and this question will be updated for you. (bcache-tools's build dependencies...

0 0
2

by

kapad

Last Updated September 23, 2015 10:01 AM

I want to know if there is anyone who has tried installing and running their kernel built with flashcache or bcache for SSD caching?

How did you manage to do it?

Answers 4

Prior warning: bcache can not be used for your existing Ubuntu installation. If you need SSD caching for your operating system you should wait for a future or special version of the Ubuntu installer which could set up your SSD cache. See this answer by Oli here: Does bcache require a fresh installation?

For everyone else who wants a bcache drive for non-OS data:

The first thing to do is upgrade your kernel if your using 13.04 or older using the kernel mainline PPA. Instructions for using this can be found here:

https://wiki.ubuntu.com/Kernel/MainlineBuilds

Make sure you select the 3.9 or higher kernel when booting or it won't work and your drive will disappear.

Install bcache-tools, at the...

0 0
3

Hey there, @kajito1!

I definitely don't see any benefits of doing this. You would be better off using the 128 GB for other software apps or data, instead of configuring it for caching. Moreover, you will see no performance benefits of a small SSD caching a larger SSD because they both operate at fast speed that is typical for solid-state drives. You could use it for the secondary HDD for 'emergencies' but then again, the idea of an SSD cache is to allow you to load faster the most frequently used files, which is a great performance boost if you were to boot from a hard drive. However, it won't make much of a difference if you set it up onto the general storage hard drive either.

Either way, don't forget to keep a copy of your files somewhere off-site! Having the HDD for 'emergencies' inside the PC rig is not really a reliable backup solution, so keeping a second copy of it somewhere else is highly recommended.

Here's another thread from the community...

0 0
4

I want to know if there is anyone who has tried installing and running their kernel built with flashcache or bcache for SSD caching?

How did you manage to do it?

Well, we got a bcache answer but no flashcache answer. I chose flashcache because I already had an existing installation, so bcache was out of the question. To me it seemed easier to set up as well. I chose the DKMS method so I wouldn't be stuck rebuilding the module/tools every time I got a kernel upgrade.

These steps are outlined in README-DKMS on Github.

Building the Module

The first thing you do is install the building prerequisites:

sudo apt-get install dkms build-essential linux-headers-$(uname -r) git

Then snag the source:

git clone git://github.com/facebook/flashcache; cd flashcache

Build and install the modules:

make -f Makefile.dkms

Creating a flashcache device

(In this example I use writeback caching -- the default -- but flashcache can do...

0 0
5

Btw, in order to setup ssd caching with a hdd, you need a motherboard with a specific Intel chipset (with Intel sata ports and not something like Marvell sata ports) and their SRT software that does clever things like put these drives in a bios raid so your OS will see 1 drive. I've had a bios raid before and I can tell you firsthand: there's going to be the occasional unforeseen problem that made you just wish that you simply bought a $500 raid card instead of dealing with "pseudo raid" bios configs. The only exception would be raid0 without parity checks, I wouldn't even run a mirror on a bios raid chip. Also, good luck with trim support on the ssd in that config -- it'll probably be a coin flip to see if it'll be enabled. And, when things go bad (as with all hdd's and ssd's eventually,) have fun trying to sort out where your data actually resides, assuming it's not in multiple chunks on both drives, oh myyyyyyy dat nightmare doe. Can you say "hourly backups?"

This would...

0 0
6

I don't have a name for this so I'll present my special user case:

I have an hybrid-laptop made of a tablet slate of only 32GB SSD and a 500GB HDD build into the USB keyboard-dock and I want to make a better use of that HDD without having it constantly spinning (I take a big hit on battery life otherwise)

Let's say I create a mount point \mnt\Data that resides on my external HDD. What I want to achieve is to have part (not all) of the files in that directory (maybe last n used\opened ones) cached inside my SSD somewhere without having me to copy them manually, and then update the old ones into the HDD once I modify them.

What I am looking for: Having 10-20 files I often/recently use cached and modify them on my SSD, rather than directly access it from HDD, allows me to work without the system spinning up my HDD (saving me a lot of power).

Even when I save my work, the HDD is not absolutely required to wakeup to update the files suddenly, this can happen...

0 0
7

Many of the following steps in this post are no longer required or recommended. Consider it more of a behind the scenes guide for dm-cache from the early days. :)

Modern distros have kernel support for dm-cache out of the box, so don’t worry about updating the kernel. The latest lvm2 tools have support for lvmcache which is a front-end to dm-cache and is much easier to use. At the time of this update, it appears that only Fedora (and derivatives) have support. Arch Linux bug 40754 and the 41291 dupe I reported will track the currently missing feature in Arch Linux. For more documentation on lvmcache consider man lvmcache if your lvm2 package is up to date and has cache support. Also consider Richard Jones’ LVM Cache blog post for a guide that approximates the man page. I’ve played with lvmcache and like it, if I have time I’ll write-up a blog post regarding performance. For reference, running git grep blah on a recent kernel tree takes 25-30 seconds on traditional HDD, 2...
0 0
8

I have used WriteBack in the past for two primary reasons:

1) Faster writes from the host perspective.

2) Reordering disk writes.

Faster writes allow the host to write to enclosure RAM and then continue on (with battery backup of course). Reordering allows these writes to take place in a different order than they were received from the host. Data can be written at leisure when the read/write heads are in close vicinity to a write location. Though I haven't read it specifically, I would speculate that some enclosures are more efficient at reordering packets and deferring writes than others, based on the understanding and skills of the team writing the firmware.

Let's compare an SSD drive with a 15k SAS drive. Using an Intel 320 for example, the specs show up to 38000 random IOPS during reads (14000 for writes), while 15k disks can reach, say 200 random IOPS during reads. That would give each SSD drive the same speed as about 190 hard drives.

Since...

0 0
9

Ubuntu Desktop online tour

The Ubuntu community on Reddit

This subreddit is for news, information and general discussion related to Ubuntu.

Getting started

Rules

Homophobia, sexism, racism or any derogatory language will not be tolerated. Please refer to the Ubuntu Code of Conduct. Tech support questions must be links to Ask Ubuntu or the forums (here's why). Posts must relate to Ubuntu (i.e. not just Linux in general). No memes or follow-ups to picture posts ("I see your Ubuntu-CDs and give you these...").

Support resources

Documentation

Flair

Ubuntu Members and employees of Canonical have emblems next to their names, indicating their affliation. Members are distinguished by a small Ubuntu logo next to their names, Canonical employees by a purple "O", a portion of Canonical's logo.

To apply for flair, please see here.

Subreddit CSS

The CSS on /r/Ubuntu is an on going development keeping up to date with...

0 0
10
As per Microsoft: "turning disk write caching on may increase operating system performance; however, it may also result in the loss of information if a power failure, equipment failure, or software failure occurs"

I would like opinions on this. I just realized that I have not been offered the choice of safety vs performance in the last couple of iterations of the Windows OS. For years I have chosen 'no cache' for HDD, selecting safer but slower. Now, for a number of years, I have not been offered the choice.

Since I do not have UPS on my systems, I feel 'no cache' is appropriate.

For those inclined towards safety, you might appreciate this link: https://support.microsoft.com/en-us/help/324805/how-to-manually-turn-disk-write-caching-on-or-off

Performance vs Safety for HDD vs SSD vs Flash Drive:...

0 0
11

There are several ways to use an SSD as a cache for your regular hard drive. This is not a new idea, and already has several implementations.

In Windows:

The best implementation is by NVelo: Dataplex - It has read caching and optional write caching. Unfortunately, like ExpressCache it is available to OEMs only. The cheapest way to get it is to buy an OCZ SSD which packages a PCIE adapter with a small SSD and uses the dataplex software to cache your existing harddrive.

There are however, pure software implementations too: FancyCache - It works like you might expect performing read and write caching at the block layer. It primarily exists as a memory cache, but recently has gained support for a second layer SSD cache (a bit like the ZFS L2ARC), which can be arbitrarily large.

If you are just looking to max out on IOPS at the cost of write durability (perfectly fine in conjunction with a UPS for example), an alternative might be something like: SuperCache -...

0 0
12

A couple of manufacturers are selling solutions to speed up your big HDD using a relative small SSD. There are techniques like:

But there is also a lot of development on the Linux front. There is Flashcache developed by Facebook, dm-cache by VISA and bcache by Kent Overstreet. The last one is interesting, because it’s a patch on top of the Linux kernel and will hopefully be accepted upstream some day.

Hardware setup

In my low power homeserver I use a 2TB Western Digital Green disk (5400 RPM). To give bcache a try I bought a 60GB Intel 330 SSD. Some facts about these data-carriers. The 2TB WD can do about 110 MB/s of sequential reads/writes. This traditional HDD does about 100 random operations per second. The 60GB Intel 330 can sequentially read about 500 MB/s and write about 450 MB/s. Random reads are done in about 42.000 operations per second, random writes in about 52.000. The SSD is much faster!

The image below shows the idea of SSD caching....

0 0
13

How to use the SSD Caching technology?


Nowadays you can use a SSD as cache (64GB max) to boost the transfer rate of a normal hard disk (Sata II or Sata III).

This feature can increase the performance of large capacity disks.



Prerequisites

SSD

You will require a SSD with a minimum capacity of 64GB (optimal size).


HDD

Use a large capacity disk - 2TB.


Motherboard

Your motherboard must support the SSD Caching Technology (Asus, MSI, ASRock, Gigabyte).



Softwares

Based on the motherboard and SSD you are using:

Intel Smart Response:


Dataplex:


ExpressCache:



Implementation

Connect two hard drives

Warning: both HDDs must be connected to the same controller, e.g Marvel or Intel ...

Configuring the Cache

The default option, named "Enhanced" is less efficient but more secure than the...

0 0
14

I have a 1TB array set up in raid 1 using mdadm on Ubuntu 8, to store mission-critical files. I'd like to speed up read/write, especially for large files by caching to an SSD. Is this possible? Does mdadm contain a facility to cache to another drive?

Alternatively, is there something I should do to assign ordinary RAM to caching? There is 4GB RAM and files to be written are rarely larger than 1GB so I'm thinking writes should be quite quick just with RAM caching alone.

Any help will be greatly appreciated.

Does mdadm contain a facility to cache to another drive?

No, the Linux software raid (managed by mdadm) is purely for creating a set of disks for redundancy purposes.1

I know of two projects that will let you do this: dm-cache and flashcache (both relatively experimental). It enables you to use your SSD as cache (read and write) for your slower hard drives or any other block device such as an md device. Both offer huge performance gain and are...

0 0
15


Question:

I know this might seem a strange question and I know I should buy a good UPS but I already had had two of them and none of them did a good job. I'm living in a house where many power outages occur (please, don't ask me why...) so data loss risk is very high. I bought a new SSD but I'm planning to keep my old HDDs, so I thought about this configuration:

/ on SSD (ext4 of course) /var on HDD /tmp on HDD /swap on HDD (swappiness set to 1 since I have 4 GB of RAM) scheduler = deadline or noop (can you point me a good way to do this permanently?) journaling = enabled (due to the power losses I can't just disable it but is it possibile to store the journal on another HDD? If yes, how?) discard and noatime in fstab

What do you think about that?

The SSD I bought is a Samsung 120 GB 840 series and my mainboard supports SATA2. Is it really needed to put var, tmp and cache on another HDD? What if I just install the whole system on my SSD, how...

0 0