alex/bcachefs-tools
1
0
Fork 0
mirror of https://github.com/koverstreet/bcachefs-tools.git synced 2025-02-10 00:00:04 +03:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,613 Commits 9 Branches 45 Tags 11 MiB
56b2449ded
Commit Graph

260 Commits

Author SHA1 Message Date
Kent Overstreet
0a08ddf78c Update bcachefs sources to b9bd69421f73 bcachefs: x-macro-ify inode flags enum 2023-11-02 14:39:01 -04:00
Kent Overstreet
9799b119c3 Update bcachefs sources to 0d63ed13ea3d closures: Fix race in closure_sync() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-10-25 13:59:16 -04:00
Kent Overstreet
2252eecec7 Update bcachefs sources to 24bdb6fed91c bcachefs: bch2_btree_id_str() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-10-19 22:56:59 -04:00
Kent Overstreet
6a34032417 Update bcachefs sources to a180af9dd349 bcachefs: Refactor memcpy into direct assignment 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-10-18 22:14:53 -04:00
Kent Overstreet
a053ebfb8c Update bcachefs sources to f9c612bbf82d bcachefs: Fixes for building in userspace 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-09-23 20:03:23 -04:00
Brian Foster
f76c449ec0 bcachefs-tools: fix endian problems between bit spinlocks and futexes 
bcachefs format on a big-endian (s390x) machine crashes down in the
rhashtable code imported from the kernel. The reason this occurs
lies within the rht_lock() -> bit_spin_lock() code, the latter of
which casts bitmaps down to 32-bits to satisfy the requirements of
the futex interface.

The specific problem here is that a 64 -> 32 bit cast doesn't refer
to the lower 8 bytes on a big endian machine, which means setting
bit number 0 in the 32-bit map actually corresponds to bit 32 in the
64-bit map. The rhashtable code specifically uses bit zero of the
bucket pointer for exclusion and uses native bitops elsewhere (i.e.
__rht_ptr()) to identify NULL pointers. If bit 32 of the pointer is
set by the locking code instead of bit 0, an otherwise NULL pointer
looks like a non-NULL value and results in a segfault.

The bit spinlock code imported by the kernel is originally intended
to work with unsigned long. The kernel code is able to throttle the
cpu directly when under lock contention, while the userspace
implementation relies on the futex primitives to emulate reliable
blocking. Since the futex interface introduces the 32-bit
requirement, isolate the associated userspace hack to that
particular code.

Restore the native bitmap behavior of the bit spinlock code to
address the rhashtable problem described above. Since this is not
compatible with the futex interface, create a futex wrapper
specifically to convert the native bitmap type to a 32-bit virtual
address and mask value for the purposes of waiting/waking when under
lock contention.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-09-19 17:26:45 -04:00
Kent Overstreet
505c326cbe Update bcachefs sources to a8115093df bcachefs: Fix divide by zero in rebalance_work() 2023-08-17 17:11:27 -04:00
Kent Overstreet
f3976e3733 Update bcachefs sources to b0788c47d9 bcachefs: Fix check_version_upgrade() 2023-08-11 22:25:12 -04:00
Kent Overstreet
87179c7a6e Update bcachefs sources to 33a60d9b05 bcachefs: Assorted fixes for clang 2023-08-01 22:13:03 -04:00
Kent Overstreet
c8bec83e30 Update bcachefs sources to e14d7c7195 bcachefs: Compression levels 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-07-15 17:36:15 -04:00
Kent Overstreet
21ae8a4b71 Update bcachefs sources to 070ec8d07b bcachefs: Snapshot depth, skiplist fields 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-07-10 14:27:14 -04:00
Kent Overstreet
5d507f795b Update bcachefs sources to da7d42a9a2 bcachefs: Add new assertions for shutdown path 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-07-09 15:25:40 -04:00
Kent Overstreet
c99a444c5c Update bcachefs sources to 25de2b00dc bcachefs: Change check for invalid key types 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-07-07 04:25:07 -04:00
Kent Overstreet
619c99de4b get_random_u32_below() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-06-27 19:45:40 -04:00
Kent Overstreet
cfa816bf3f Update bcachefs sources to 84f132d569 bcachefs: fsck: Break walk_inode() up into multiple functions 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-06-27 19:38:18 -04:00
Kent Overstreet
0708303f49 Switch to CLOCK_MONOTONIC_COARSE 
CLOCK_MONOTONIC requires a syscall, and is much more expensive.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-06-18 14:32:28 -04:00
Kent Overstreet
17d1c4f4fe Update bcachefs sources to ed6b7f81a7 six locks: Disable percpu read lock mode in userspace 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-06-10 22:15:00 -04:00
Kent Overstreet
8642d4ae10 Update bcachefs sources to 7c0fe6f104 bcachefs: Fix bch2_fsck_ask_yn() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-06-04 18:10:29 -04:00
Kent Overstreet
1f78fed469 Update bcachefs sources to 31c09369cd six locks: Fix an unitialized var 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-05-25 22:25:34 -04:00
Kent Overstreet
5ef62f56ab Update bcachefs sources to 400f275d46 bcachefs: Fix check_overlapping_extents() 2023-05-13 00:13:57 -04:00
Kent Overstreet
6b1f79d5df Update bcachefs sources to 6a20aede29 bcachefs: Fix quotas + snapshots 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-05-02 00:41:47 -04:00
Mikulas Patocka
4f6b28f54f A small compile fix 
Hi

Here I'm sending a small compile fix for bcachefs-tools.

Without this patch, I get this error:

cargo  build --release --manifest-path rust-src/Cargo.toml
   Compiling bch_bindgen v0.1.0
(/usr/src/git/bcachefs-tools/rust-src/bch_bindgen)
error: failed to run custom build command for `bch_bindgen v0.1.0
(/usr/src/git/bcachefs-tools/rust-src/bch_bindgen)`

Caused by:
  process didn't exit successfully:
`/usr/src/git/bcachefs-tools/rust-src/target/release/build/bch_bindgen-733e88995ce9eab7/build-script-build`
(exit status: 101)
  --- stderr
  warning: optimization flag '-fkeep-inline-functions' is not supported
[-Wignored-optimization-argument]
  ../../include/linux/bit_spinlock.h:20:3: error: call to undeclared
function 'futex'; ISO C99 and later do not support implicit function
declarations [-Wimplicit-function-declaration]
  ../../include/linux/bit_spinlock.h:28:2: error: call to undeclared
function 'futex'; ISO C99 and later do not support implicit function
declarations [-Wimplicit-function-declaration]
  ../../include/linux/bit_spinlock.h:39:2: error: call to undeclared
function 'futex'; ISO C99 and later do not support implicit function
declarations [-Wimplicit-function-declaration]

The futex() function is declared in
	/usr/include/x86_64-linux-gnu/urcu/futex.h
It is not declared in linux/futex.h, so we need to include urcu/futex.h

Mikulas

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-04-26 10:44:49 -04:00
Kent Overstreet
fbe4e11c26 Update bcachefs sources to 2115a2ffde bcachefs: Kill bch2_verify_bucket_evacuated() 2023-04-21 04:02:48 -04:00
Kent Overstreet
807b250927 Update bcachefs sources to 504729f99c bcachefs: Allow answering y or n to all fsck errors of given type 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-04-15 14:41:11 -04:00
Kent Overstreet
7f102ee83d Update bcachefs sources to 8fd009dd76 bcachefs: Rip out code for storing backpointers in alloc keys 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-03-31 18:02:57 -04:00
Kent Overstreet
0206d42daf Update bcachefs sources to 3856459b1b bcachefs: bch2_btree_iter_peek_node_and_restart() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-03-06 04:11:50 -05:00
Kent Overstreet
b0c9ad15f4 Update bcachefs sources to 171da96d76 bcachefs: Drop some anonymous structs, unions 2023-03-04 23:33:25 -05:00
Kent Overstreet
d010a21653 linux shim: hlist_unhashed() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-03-02 23:00:03 -05:00
Kent Overstreet
30cca2e94d Update bcachefs sources to ca97ee3577 bcachefs: bch2_btree_iter_peek_and_restart_outlined() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-02-27 21:36:36 -05:00
Kent Overstreet
da6a356895 Update bcachefs sources to 8e1519ccb6 bcachefs: Add tracepoint & counter for btree split race 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-02-20 20:03:34 -05:00
Kent Overstreet
46a6b9210c Update bcachefs sources to 1b14994029 bcachefs: Fragmentation LRU 2023-02-17 17:51:22 -05:00
Kent Overstreet
c1677df62e Fix cpu_relax() 
cpu_relax() is supposed to be a compiler barrier - this fixes a bug with
btree_write_buffer_flush() getting stuck.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-02-13 17:51:14 -05:00
Kent Overstreet
e160e9b979 Update bcachefs sources to 3e0c5b0722 fixup! bcachefs: Btree write buffer 2023-02-12 21:41:03 -05:00
Kent Overstreet
a104f0407b Update bcachefs sources to ea93c26e98 fixup! bcachefs: We can handle missing btree roots for all alloc btrees 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2023-02-09 18:36:24 -05:00
Kent Overstreet
e2670a38d1 Change memory reclaim 
- Spin up a background thread to call the shrinkers every 1 second
 - Memory allocations will only call reclaim after a failed allocation,
   not every single time

This will be a major performance boost on allocation intensive
workloads.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2022-12-19 14:47:42 -05:00
Kent Overstreet
3c0484687d Update bcachefs sources to 84505cfd37 bcachefs: Go RW before check_alloc_info() 2022-12-13 18:27:31 -05:00
Kent Overstreet
a06dee6da2 Update bcachefs sources to ed2a5f4260 bcachefs: Add a missing bch2_btree_path_traverse() call 2022-12-09 16:40:07 -05:00
Kent Overstreet
934a84dfaf Update bcachefs sources to 5963d1b1a4 bcacehfs: Fix bch2_get_alloc_in_memory_pos() 2022-11-26 17:09:59 -05:00
Stijn Tintel
48eefee749 Fix build against musl libc. 
The musl C library does not define __attribute_const__. Add it to
include/linux/compiler.h with a guard to fix build against musl libc.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
 2022-11-24 06:21:06 +02:00
Kent Overstreet
c1e4d447f6 Update bcachefs sources to 8d3fc97ca3 bcachefs: Fixes for building in userspace 2022-11-13 20:07:22 -05:00
Kent Overstreet
ae43a58d97 Update bcachefs sources to 70fa0c1ff4 fixup! bcachefs: Btree key cache improvements 2022-10-23 18:24:08 -04:00
Kent Overstreet
188b6d0c8e Update bcachefs sources to cd779e0cc5 bcachefs: Skip inode unpack/pack in bch2_extent_update() 2022-10-22 14:41:54 -04:00
Kent Overstreet
e0a51ccce8 Update bcachefs sources to 3e93567c51 bcachefs: Switch to local_clock() for fastpath time source 2022-10-15 01:04:30 -04:00
Kent Overstreet
0398adf20f fixup! Don't run shrinkers without GFP_KERNEL 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2022-10-11 03:22:59 -04:00
Kent Overstreet
81aef33957 Don't run shrinkers without GFP_KERNEL 
This would correspond to GFP_RECLAIM in the kernel - but we don't
distinguish between different types of reclaim here.

This solves a deadlock in the btree node memory allocation path - we
allocate with the btree node cache lock held but without GFP_KERNEL set.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2022-10-11 00:39:06 -04:00
Kent Overstreet
72add8822c Update bcachefs sources to 47ffed9fad bcachefs: bch2_btree_delete_range_trans() now uses peek_upto() 2022-10-10 00:10:45 -04:00
Kent Overstreet
8d6138baac Update bcachefs sources to cbccc6d869 bcachefs: Ratelimit ec error message 2022-10-09 02:33:05 -04:00
Kent Overstreet
99caca2c70 Update bcachefs sources to 24c6361e20 bcachefs: Fix a trans path overflow in bch2_btree_delete_range_trans() 
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
 2022-09-28 11:41:03 -04:00
Kent Overstreet
d01f633041 Update bcachefs sources to 176718966e bcachefs: Re-enable hash_redo_key() 2022-09-04 14:42:05 -04:00
Kent Overstreet
dded444b20 Update bcachefs sources to dfaf9a6ee2 lib/printbuf: Clean up headers 2022-08-18 12:49:48 -04:00
Kent Overstreet
ddac1641ee Update bcachefs sources to 90a9c61e2b bcachefs: Switch bch2_btree_delete_range() to bch2_trans_run() 2022-08-11 17:34:34 -04:00
Kent Overstreet
79d39bd7ce Update bcachefs sources to d82da7126f fixup! bcachefs: for_each_btree_key2() 2022-07-17 05:20:54 -04:00
Kent Overstreet
bad0c8c507 Update bcachefs sources to 95ff72a6c1 fixup! mm: Centralize & improve oom reporting in show_mem.c 2022-06-27 15:58:48 -04:00
Kent Overstreet
64ddfc9fc5 hweight32() 
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-06-27 15:58:39 -04:00
Kent Overstreet
23dc00de78 Fix printk_ratelimited() 
printk_ratelimited was behind an #ifdef CONFIG_PRINTK, which we don't
define, so it was a complete noop - oops.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-06-22 22:05:39 -04:00
Kent Overstreet
03b8098d37 cmd_dump: Use buffered IO for reading device metadata 
We're reading device metadat in mostly sequential order - buffered IO
will be faster than O_DIRECT.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-06-18 18:33:25 -04:00
Kent Overstreet
ffbf806d5c Update bcachefs sources to 3704d0779c bcachefs: Improved human readable integer parsing 2022-06-11 18:14:06 -04:00
Kent Overstreet
6ac37db841 Update bcachefs sources to 24f7e08cd8 bcachefs: shrinker.to_text() methods 2022-06-09 15:07:25 -04:00
Kent Overstreet
46b2c553aa Update bcachefs sources to fad6d13aa5 fixup! bcachefs: Add persistent counters 2022-05-30 23:24:10 -04:00
Kent Overstreet
6f5afc0c12 Update bcachefs sources to bdf6d7c135 fixup! bcachefs: Kill journal buf bloom filter 2022-05-02 18:42:06 -04:00
Kent Overstreet
04ad4ade4c Update bcachefs sources to fb8a27f6d4 bcachefs: Delete a redundant tracepoint 2022-04-11 22:40:16 -04:00
Kent Overstreet
986533d8d5 Update bcachefs sources to 6ddf061e68 bcachefs: Use a genradix for reading journal entries 2022-04-08 19:26:30 -04:00
Kent Overstreet
498874fdb7 Update bcachefs sources to 91e6c3e0d5 bcachefs: Gap buffer for journal keys 2022-04-04 13:48:45 -04:00
Kent Overstreet
32aabbc4e5 Update bcachefs sources to ab3b6e7dd6 bcachefs: Fix pr_buf() calls 2022-04-02 16:36:29 -04:00
Kent Overstreet
70f2681838 Update bcachefs sources to 5e392aed7a bcachefs: Kill bch2_alloc_write() 2022-03-31 16:57:08 -04:00
Kent Overstreet
cc1b64e992 Flush standard output before popping assertions 
Seeing error messages before we die generally makes assertions easier to
debug.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-03-30 23:20:39 -04:00
Kent Overstreet
e240b4ae86 Update bcachefs sources to f638850417 bcachefs: bch2_trans_log_msg() 2022-03-30 17:40:37 -04:00
Kent Overstreet
1a9d73b5c0 Shrinker improvements 
After memory allocation failure, don't rely on /proc/meminfo to figure
out how much memory we should free - instead unconditionally free 1/8th
of each cache.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-03-21 20:20:09 -04:00
Kent Overstreet
205d75307a Better bio_alloc_bioset() 
bio_alloc_bioset() now uses mempools, so we don't segfault on memory
allocation failure.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-03-21 18:21:31 -04:00
Kent Overstreet
74148a8ee5 Update bcachefs sources to 0e705f5944 fixup! bcachefs: Refactor bch2_btree_node_mem_alloc() 2022-03-21 02:13:02 -04:00
Kent Overstreet
3765483ff0 Update bcachefs sources to f05b3c1af9 bcachefs: Improve bucket_alloc_fail tracepoint 2022-03-13 19:21:13 -04:00
Kent Overstreet
d34e731082 Update bcachefs sources to e48731a188 bcachefs: Fix BTREE_TRIGGER_WANTS_OLD_AND_NEW 2022-03-13 19:20:59 -04:00
Kent Overstreet
bf7924f552 Add freezable_schedule() to linux kernel shim 
Also fix the freezable_schedule_timeout() macro.

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-03-10 15:38:02 -05:00
Kent Overstreet
03498f9464 Update bcachefs sources to 04036b4910 bcachefs: Fix a memory leak 2022-02-27 12:01:32 -05:00
Kent Overstreet
8d1c88c30d Update bcachefs sources to da8056a215 bcachefs: Store logical location of journal entries 2022-02-19 01:31:29 -05:00
Kent Overstreet
cc1a99fb74 Switch to pthread_mutex_t for spinlocks, too 
This fixes an observed journal deadlock - real spinlocks aren't safe in
userspace.
 2022-02-18 05:58:24 -05:00
Kent Overstreet
de4e778a6a Update bcachefs sources to a0d7001b0f bcachefs: Change bch2_dev_lookup() to not use lookup_bdev() 2022-02-16 06:32:49 -05:00
Kent Overstreet
a1d66a2a4e Update bcachefs sources to ba398d2906 bcachefs: Fix reflink repair code 2022-02-13 04:08:33 -05:00
Kent Overstreet
9c79275419 Update bcachefs sources to b84661c042 bcachefs: Fix reflink repair code 2022-02-11 18:33:16 -05:00
Kent Overstreet
74a7c3ff57 Update bcachefs sources to bf340e68c7 bcachefs: Ignore cached data when calculating fragmentation 2022-01-10 21:41:51 -05:00
Kent Overstreet
931ed5a709 Update bcachefs sources to 50ac18afbb bcachefs: Fix an uninitialized variable 2022-01-04 19:56:40 -05:00
Kent Overstreet
617dc6dd68 Retry memory allocation failures 
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2022-01-01 21:14:28 -05:00
Kent Overstreet
38f8daa2b1 Update bcachefs sources to 078a1a596a bcachefs: Optimize bucket reuse 2021-12-28 15:03:54 -05:00
Kent Overstreet
42ba8b5a75 Update bcachefs sources to 5fd0c70102 bcachefs: Fix __remove_dirent() 2021-11-03 21:28:23 -04:00
Brett Holman
76821f0760 make BIO_MAX_VECS unsigned, like the kernel definition 
Signed-off-by: Brett Holman <bholman.devel@gmail.com>
 2021-10-16 13:41:43 -06:00
Kent Overstreet
2b8c1bb091 Update bcachefs sources to 3f3f969859 bcachefs: Fix some compiler warnings 2021-09-09 19:10:07 -04:00
Kent Overstreet
ac82bc1ea5 Delete unused variable 
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2021-07-28 17:27:25 -04:00
Kent Overstreet
f06b01e9ea Update bcachefs sources to b964c6cba8 bcachefs: Change lockrestart_do() to always call bch2_trans_begin() 2021-07-28 17:26:50 -04:00
Brett Holman
308d7adcc5 fix for musl and non-x86 archs 2021-07-24 20:16:41 -04:00
Kent Overstreet
86bd5c622c Update bcachefs sources to 787de128a5 bcachefs: Improvements to fsck check_dirents() 2021-07-14 20:31:38 -04:00
Kent Overstreet
700d013b52 Update bcachefs sources to 400c2f8d96 bcachefs: Mask out unknown compat features when going read-write 2021-07-11 13:57:11 -04:00
Kent Overstreet
347c0b108e Update bcachefs sources to dbee44d5ab bcachefs: add bcachefs xxhash support 2021-07-07 20:30:53 -04:00
Kent Overstreet
ff578c6718 Update bcachefs sources to 69be0dae31 bcachefs: Always zero memory from bch2_trans_kmalloc() 2021-06-10 20:25:55 -04:00
Kent Overstreet
58ae103b0a Update bcachefs sources to e3a7cee503 bcachefs: Don't mark superblocks past end of usable space 2021-06-08 22:55:14 -04:00
Kent Overstreet
b61ad35b97 Update bcachefs sources to 3913e0cac3 bcachefs: Journal space calculation fix 2021-05-31 15:05:33 -04:00
Kent Overstreet
4a2acdaf65 Update closures from kernel source tree 2021-05-27 21:13:52 -04:00
Kent Overstreet
27d75d929f Grab math.h from kernel source tree 
This gets us round_up() and round_down()

Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2021-05-27 21:12:13 -04:00
Kent Overstreet
2e2d5a3f7e Update bcachefs sources to 0cd3e1d27a bcachefs: Fix for bch2_bkey_pack_pos() not initializing len/version fields 2021-05-17 16:48:48 -04:00
Kent Overstreet
8f72e6940c Fix some minor compiler warnings 
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
 2021-05-17 16:35:54 -04:00
Kent Overstreet
bb74624daa Update bcachefs sources to a5c0e1bb30 bcachefs: Clean up bch2_btree_and_journal_walk() 2021-04-30 18:23:38 -04:00