Commit Graph

1192 Commits

Author SHA1 Message Date
Kent Overstreet
e3e7f67b3e Merge remote-tracking branch 'amoz/devel' 2023-12-07 20:19:06 -05:00
koverstreet
36898f574b
Merge pull request #190 from Dikay900/fs_free_space
cmd_fs: calculate free space
2023-12-07 20:10:15 -05:00
Alexander Fougner
244046945a fix build for glibc prior to 2.33
Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-12-07 21:36:47 +01:00
Alexander Fougner
262e548a84 fix fallback flex array
Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-12-07 21:36:45 +01:00
Philip Laube
5bc5fc621a cmd_fs: calculate size based on bucket count for data types BCH_DATA_free, BCH_DATA_need_discard and BCH_DATA_need_inc_gens
Signed-off-by: Philip Laube <philip.laube.dev@xyz.de>
2023-12-07 20:47:09 +01:00
Kent Overstreet
ea5264fdf4 Online fsck
This hooks up cmd_fsck to the new BCH_IOCTL_FSCK_ONLINE: if a filesystem
is mounted, we now use that instead of bailing out with an error.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-07 12:04:24 -05:00
Kent Overstreet
8d8a9f3e9b Update bcachefs sources to 6d44812757dd bcachefs: BCH_IOCTL_FSCK_ONLINE
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-07 11:57:36 -05:00
Kent Overstreet
db9ae9a79d Update .gitignore for new service files
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-07 11:57:36 -05:00
Darrick J. Wong
74753de989 fsck: add a service to schedule and run bcachefsck@ services
Create a script that looks for mounted bcachefs filesystems and tries to
run as many of them in parallel as possible without two different
scrubbers hitting the same low level block device.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
2023-12-07 11:57:36 -05:00
Darrick J. Wong
fbfdd05ac5 fsck: add systemd service definitions for automatic online service
Add some systemd service files so that bcachefs can automatically fsck
mounted filesystems in the background.  Hopefully with minimal
disruption to frontend operations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
2023-12-07 11:57:36 -05:00
koverstreet
5fa7db9806
Merge pull request #191 from cepelinas9000/fuse-fixes-1
Multiple small fuse fixes for write, api, and hardlink
2023-12-07 11:57:04 -05:00
Tautvydas Nesvarbu
bb50e50173 Multiple small fuse fixes for write, api, and hardlink
* changed in Makefile to use recomended fuse api 35;
* force foreground mode, because fuse_daemonize is forking after initalization other bcachefs threads;
* fix fuse_link, there where mistype in fuse_log (the paremeter need to be inum.inum);
* The write_aligned is fixed, by  wrapping struct bch_write_op into container with single closure and using end_io callback to release
      that closure.
2023-12-07 15:17:34 +02:00
Alexander Fougner
1c2cb80376 fix udev rules path
- include rules file in rpmspec
- update install docs with udev build requirement

Signed-off-by: Alexander Fougner <fougner89@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-06 15:41:05 -05:00
koverstreet
5d892680cf
Merge pull request #189 from Dikay900/fs_human_readable
cmd_fs: add long parameter --human-readable to usage command
2023-12-06 13:58:01 -05:00
Philip Laube
ec028b4913 cmd_fs: add long parameter --human-readable to usage command
Signed-off-by: Philip Laube <philip.laube.dev@xyz.de>
2023-12-06 19:29:05 +01:00
Kent Overstreet
cf78a3f65c Disable shrinker thread shutdown
We seem to be hitting a rare crash in the exit path of fsck - when
shutting down the shrinker thread. Disable exiting the shrinker thread
as a workaround.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-05 23:53:29 -05:00
Darrick J. Wong
f407121d18 misc: don't allow udisks to automount bcachefs filesystems with no prompt
The unending stream of syzbot bug reports and overwrought filing of CVEs
for corner case handling (i.e. things that distract from actual user
complaints) in XFS has generated all sorts of of overheated rhetoric
about how every bug is a Serious Security Issue(tm) because anyone can
craft a malicious filesystem on a USB stick, insert the stick into a
victim machine, and mount will trigger a bug in the kernel driver that
leads to some compromise or DoS or something.

I thought that nobody would be foolish enough to automount an XFS
filesystem.  What a fool I was!  It turns out that udisks can be told
that it's okay to automount things, and then GNOME will do exactly that.
Including mounting mangled XFS filesystems!  Same with bcachefs!

<delete angry rant about poor decisionmaking and armchair fs developers
blasting us on X while not actually doing any of the work>

Turn off /this/ idiocy by adding a udev rule to tell udisks not to
automount bcachefs filesystems.

This will not stop a logged in user from unwittingly inserting a
malicious storage device and pressing [mount] and getting breached.
This is not a substitute for a thorough audit.  This is not a substitute
for lklfuse.  This does not solve the general problem of in-kernel fs
drivers being a huge attack surface.  I just want to give Kent a break
from some of the oceans of bu******.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-12-04 19:49:26 -05:00
Kent Overstreet
f27b135285 Update bcachefs sources to 71a5b27e017d bcachefs: Make backpointer fsck wb flush check more rigorous
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-30 02:20:51 -05:00
Kent Overstreet
3bd4653767 Update bcachefs sources to 676dd269f0f8 mean and variance: Promote to lib/math 2023-11-29 22:42:52 -05:00
Kent Overstreet
46d5160869 Update bcachefs sources to eb83f1f842bb mean and variance: Promote to lib/math 2023-11-29 22:00:41 -05:00
Kent Overstreet
314ce7d963 add a fallback __DECLARE_FLEX_ARRAY
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-29 17:41:06 -05:00
Kent Overstreet
89b361f24a Update bcachefs sources to c3e4d892b77b mean and variance: Promote to lib/math 2023-11-29 17:00:30 -05:00
Brett Holman
1dc5842111 man: Fix cmd-option format
Options prgquote and degraded only had a single leading hyphen.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27 21:04:03 -05:00
Brian Foster
1cdb7812d1 bcachefs-tools: don't skip stale superblock wipe in force mode
Even though the blkid wipe has been fixed to detect and wipe old
superblocks, we still incorrectly skip the wipe when in force mode.
Update the force logic in open_for_format() to bypass the user
request and otherwise proceed with the wipe.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27 21:04:03 -05:00
Kent Overstreet
843707ede6 cmd_list_journal: Apply star_start_of_lines() correctly
In list_journal, we note which journal entries are being ignored - this
was forgetting to mark entries newer than the newest flush.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27 21:04:03 -05:00
Kent Overstreet
97c86db4f2 Update bcachefs sources to feaca6edbd24 mean and variance: Promote to lib/math
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27 21:04:03 -05:00
Kent Overstreet
7465d1934c Convert to BCH_IOCTL_DEV_USAGE_V2
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27 21:04:03 -05:00
Kent Overstreet
3a0cc86e76 Update bcachefs sources to 8c94740b1bf8 bcachefs: Add missing vaidation for jset_entry_data_usage 2023-11-25 21:51:30 -05:00
Kent Overstreet
138397d892 cmd_fs_usage: Print out replica set durability
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-25 00:06:32 -05:00
Alexander Batischev
446ec6b092 docs: mention that libclang is required
This fixes the following build failure on Debian bookworm:

    error: failed to run custom build command for `clang-sys v1.6.1`

    Caused by:
      process didn't exit successfully: `/home/minoru/src/bcachefs-tools/rust-src/target/release/build/clang-sys-df95f6d1266be773/build-script-build` (exit status: 101)
      --- stdout
      cargo:warning=could not execute `llvm-config` one or more times, if the LLVM_CONFIG_PATH environment variable is set to a full path to valid `llvm-config` executable it will be used to try to find an instance of `libclang` on your system: "couldn't execute `llvm-config --prefix` (path=llvm-config) (error: No such file or directory (os error 2))"

      --- stderr
      thread 'main' panicked at /home/minoru/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.6.1/build/dynamic.rs:206:45:
      called `Result::unwrap()` on an `Err` value: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
      note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-24 20:09:46 -05:00
Daniel Hill
326d7c1b3b improve kmalloc performance
Reading from /proc/meminfo is really slow

We don't want to start swapping to disk.

Deceptively, memory available goes up when we start to swap to disk making
performance even worse.

To mitigate this:
1. replace reading from meminfo with proper system calls.
2. attempt to lock allocations in physical memory space.
3. check our own allocated memory instead of available memory.
4. still check available memory in the off chance we're trying to play
nice with other apps.

Signed-off-by: Daniel Hill <daniel@gluo.nz>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-24 19:54:11 -05:00
Alexander Fougner
fbad1bfdf3 rust: fix MSRV build
- downgrade clap_completion to 4.3.x (1.65 compatible)
- dependency updates
- remove unused imports

Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-11-23 19:29:43 +01:00
Kent Overstreet
06611a71a3 Update bcachefs sources to 783085c3cc44 kbuild: Allow gcov to be enabled on the command line
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-22 16:55:06 -05:00
Finn Behrens
e6b578917f nix: add bcachefs-fuse package
Add a fuseSupport argument and bcachefs-fuse as flake output with fuse enabled.

Signed-off-by: Finn Behrens <me@kloenk.de>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-20 15:13:17 -05:00
Zhai Can
b09d3a3356 fix manpage date
Signed-off-by: Zhai Can <bczhc0@126.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18 14:53:17 -05:00
Zhai Can
07d4e0e864 manpage: add options for set-option
Signed-off-by: Zhai Can <bczhc0@126.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18 14:53:17 -05:00
Zhai Can
0b5a8c239b update manpage
Signed-off-by: Zhai Can <bczhc0@126.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18 14:53:17 -05:00
Kent Overstreet
c6e6954928 Fix build for cmd_fusemount.c
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-17 17:04:46 -05:00
Brian Foster
83bc15eefe bcachefs-tools: fix broken libblkid superblock wipe
When playing around with comparing some fstests results on different
filesystems, I noticed that a 'bcachefs format' of a previously
btrfs-formatted device still continued to mount as btrfs. The reason
for this is that the blkid wipe invoked via open_for_format() is not
working correctly. blkid_do_wipe() depends on the "SBMAGIC[_OFFSET]"
values to do any work, and the associated superblock magic flag is
not enabled on the probe.

Set the probe flags to explicitly enable the values the bcachefs
code depends on in the probe. This includes the type, label and
superblock magic information.

There are also a couple quirks in the libblkid code that might be
worth noting. One is that the superblock enablement and flag setting
functions appear hardcoded to return zero, so we just combine the
error checks. Second, while blkid_do_wipe() can return an error, it
actually doesn't in the scenario being addressed here because it
doesn't seem to distinguish between the values being absent because
nothing was found by the probe or because the values weren't enabled
in the first place. Regardless, add an error check here in the event
the wipe does explicitly fail for some unexpected reason.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-17 15:09:27 -05:00
Kent Overstreet
1331759db5 Bump rust edition to 2021
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16 17:54:06 -05:00
Kent Overstreet
119d26bb10 delete dead code
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16 17:54:06 -05:00
Kent Overstreet
689b4dd4f0 Update bcachefs sources to 938f680845d1 fixup! rename and export __kern_path_locked() 2023-11-16 17:54:06 -05:00
Kent Overstreet
9776f9f362 Create COPYING.rust-dependencies in release tarballs
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16 17:54:06 -05:00
Kent Overstreet
47bb2d53ce Version is now specified in the makefile
git describe doesn't work for tarball releases

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16 15:24:22 -05:00
Kent Overstreet
7fd6c3ffe4 Update bcachefs sources to 3ca08ab51ec9 bcachefs: six locks: Simplify optimistic spinning
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-12 20:57:28 -05:00
Finn Behrens
a613340b26 rust keylocation add none variant and implement ValueEnum
This enables a possible values help in the clap help text.

Signed-Off-By: Finn Behrens <me@kloenk.de>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-12 11:14:16 -05:00
Zhai Can
73da05d983
rust-cli: add help on list command 2023-11-10 21:20:17 +08:00
Zhai Can
1d1fe7b0b6
add command to generate Rust-part CLI completions 2023-11-10 21:20:01 +08:00
Finn Behrens
61134a06fa nix: add devShell
Add a nix devShell output to provide a development environment for nix users.

Signed-off-by: Finn Behrens <me@kloenk.de>
2023-11-09 23:13:43 +01:00
Finn Behrens
813310c343 nix: fix build add overlay and formatter to flake
Adds overlay flake output to expose the bcachefs package
and add and run a nix formatter.

Co-authored-by: Leona Maroni <dev@leona.is>
Signed-off-by: Leona Maroni <dev@leona.is>
Signed-off-by: Finn Behrens <me@kloenk.de>
2023-11-09 22:09:33 +01:00