Commit 3942c07c authored by Glauber Costa's avatar Glauber Costa Committed by Al Viro

fs: bump inode and dentry counters to long

This series reworks our current object cache shrinking infrastructure in
two main ways:

 * Noticing that a lot of users copy and paste their own version of LRU
   lists for objects, we put some effort in providing a generic version.
   It is modeled after the filesystem users: dentries, inodes, and xfs
   (for various tasks), but we expect that other users could benefit in
   the near future with little or no modification.  Let us know if you
   have any issues.

 * The underlying list_lru being proposed automatically and
   transparently keeps the elements in per-node lists, and is able to
   manipulate the node lists individually.  Given this infrastructure, we
   are able to modify the up-to-now hammer called shrink_slab to proceed
   with node-reclaim instead of always searching memory from all over like
   it has been doing.

Per-node lru lists are also expected to lead to less contention in the lru
locks on multi-node scans, since we are now no longer fighting for a
global lock.  The locks usually disappear from the profilers with this

Although we have no official benchmarks for this version - be our guest to
independently evaluate this - earlier versions of this series were
performance tested (details at yielding no
visible performance regressions while yielding a better qualitative
behavior in NUMA machines.

With this infrastructure in place, we can use the list_lru entry point to
provide memcg isolation and per-memcg targeted reclaim.  Historically,
those two pieces of work have been posted together.  This version presents
only the infrastructure work, deferring the memcg work for a later time,
so we can focus on getting this part tested.  You can see more about the
history of such work at

Dave Chinner (18):
  dcache: convert dentry_stat.nr_unused to per-cpu counters
  dentry: move to per-sb LRU locks
  dcache: remove dentries from LRU before putting on dispose list
  mm: new shrinker API
  shrinker: convert superblock shrinkers to new API
  list: add a new LRU list type
  inode: convert inode lru list to generic lru list code.
  dcache: convert to use new lru list infrastructure
  list_lru: per-node list infrastructure
  shrinker: add node awareness
  fs: convert inode and dentry shrinking to be node aware
  xfs: convert buftarg LRU to generic code
  xfs: rework buffer dispose list tracking
  xfs: convert dquot cache lru to list_lru
  fs: convert fs shrinkers to new scan/count API
  drivers: convert shrinkers to new count/scan API
  shrinker: convert remaining shrinkers to count/scan API
  shrinker: Kill old ->shrink API.

Glauber Costa (7):
  fs: bump inode and dentry counters to long
  super: fix calculation of shrinkable objects for small numbers
  list_lru: per-node API
  vmscan: per-node deferred work
  i915: bail out earlier when shrinker cannot acquire mutex
  hugepage: convert huge zero page shrinker to new shrinker API
  list_lru: dynamically adjust node arrays

This patch:

There are situations in very large machines in which we can have a large
quantity of dirty inodes, unused dentries, etc.  This is particularly true
when umounting a filesystem, where eventually since every live object will
eventually be discarded.

Dave Chinner reported a problem with this while experimenting with the
shrinker revamp patchset.  So we believe it is time for a change.  This
patch just moves int to longs.  Machines where it matters should have a
big long anyway.
Signed-off-by: default avatarGlauber Costa <>
Cc: Dave Chinner <>
Cc: "Theodore Ts'o" <>
Cc: Adrian Hunter <>
Cc: Al Viro <>
Cc: Artem Bityutskiy <>
Cc: Arve Hjønnevåg <>
Cc: Carlos Maiolino <>
Cc: Christoph Hellwig <>
Cc: Chuck Lever <>
Cc: Daniel Vetter <>
Cc: Dave Chinner <>
Cc: David Rientjes <>
Cc: Gleb Natapov <>
Cc: Greg Thelen <>
Cc: J. Bruce Fields <>
Cc: Jan Kara <>
Cc: Jerome Glisse <>
Cc: John Stultz <>
Cc: KAMEZAWA Hiroyuki <>
Cc: Kent Overstreet <>
Cc: Kirill A. Shutemov <>
Cc: Marcelo Tosatti <>
Cc: Mel Gorman <>
Cc: Steven Whitehouse <>
Cc: Thomas Hellstrom <>
Cc: Trond Myklebust <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarAl Viro <>
parent da5338c7
......@@ -146,13 +146,13 @@ struct dentry_stat_t dentry_stat = {
.age_limit = 45,
static DEFINE_PER_CPU(unsigned int, nr_dentry);
static DEFINE_PER_CPU(long, nr_dentry);
#if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
static int get_nr_dentry(void)
static long get_nr_dentry(void)
int i;
int sum = 0;
long sum = 0;
sum += per_cpu(nr_dentry, i);
return sum < 0 ? 0 : sum;
......@@ -162,7 +162,7 @@ int proc_nr_dentry(ctl_table *table, int write, void __user *buffer,
size_t *lenp, loff_t *ppos)
dentry_stat.nr_dentry = get_nr_dentry();
return proc_dointvec(table, write, buffer, lenp, ppos);
return proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
......@@ -70,33 +70,33 @@ EXPORT_SYMBOL(empty_aops);
struct inodes_stat_t inodes_stat;
static DEFINE_PER_CPU(unsigned int, nr_inodes);
static DEFINE_PER_CPU(unsigned int, nr_unused);
static DEFINE_PER_CPU(unsigned long, nr_inodes);
static DEFINE_PER_CPU(unsigned long, nr_unused);
static struct kmem_cache *inode_cachep __read_mostly;
static int get_nr_inodes(void)
static long get_nr_inodes(void)
int i;
int sum = 0;
long sum = 0;
sum += per_cpu(nr_inodes, i);
return sum < 0 ? 0 : sum;
static inline int get_nr_inodes_unused(void)
static inline long get_nr_inodes_unused(void)
int i;
int sum = 0;
long sum = 0;
sum += per_cpu(nr_unused, i);
return sum < 0 ? 0 : sum;
int get_nr_dirty_inodes(void)
long get_nr_dirty_inodes(void)
/* not actually dirty inodes, but a wild approximation */
int nr_dirty = get_nr_inodes() - get_nr_inodes_unused();
long nr_dirty = get_nr_inodes() - get_nr_inodes_unused();
return nr_dirty > 0 ? nr_dirty : 0;
......@@ -109,7 +109,7 @@ int proc_nr_inodes(ctl_table *table, int write,
inodes_stat.nr_inodes = get_nr_inodes();
inodes_stat.nr_unused = get_nr_inodes_unused();
return proc_dointvec(table, write, buffer, lenp, ppos);
return proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
......@@ -121,7 +121,7 @@ extern void inode_add_lru(struct inode *inode);
extern void inode_wb_list_del(struct inode *inode);
extern int get_nr_dirty_inodes(void);
extern long get_nr_dirty_inodes(void);
extern void evict_inodes(struct super_block *);
extern int invalidate_inodes(struct super_block *, bool);
......@@ -55,11 +55,11 @@ struct qstr {
#define hashlen_len(hashlen) ((u32)((hashlen) >> 32))
struct dentry_stat_t {
int nr_dentry;
int nr_unused;
int age_limit; /* age in seconds */
int want_pages; /* pages requested by system */
int dummy[2];
long nr_dentry;
long nr_unused;
long age_limit; /* age in seconds */
long want_pages; /* pages requested by system */
long dummy[2];
extern struct dentry_stat_t dentry_stat;
......@@ -1271,12 +1271,12 @@ struct super_block {
struct list_head s_mounts; /* list of mounts; _not_ for fs use */
/* s_dentry_lru, s_nr_dentry_unused protected by dcache.c lru locks */
struct list_head s_dentry_lru; /* unused dentry lru */
int s_nr_dentry_unused; /* # of dentry on lru */
long s_nr_dentry_unused; /* # of dentry on lru */
/* s_inode_lru_lock protects s_inode_lru and s_nr_inodes_unused */
spinlock_t s_inode_lru_lock ____cacheline_aligned_in_smp;
struct list_head s_inode_lru; /* unused inode lru */
int s_nr_inodes_unused; /* # of inodes on lru */
long s_nr_inodes_unused; /* # of inodes on lru */
struct block_device *s_bdev;
struct backing_dev_info *s_bdi;
......@@ -49,9 +49,9 @@ struct files_stat_struct {
struct inodes_stat_t {
int nr_inodes;
int nr_unused;
int dummy[5]; /* padding for sysctl ABI compatibility */
long nr_inodes;
long nr_unused;
long dummy[5]; /* padding for sysctl ABI compatibility */
......@@ -1471,14 +1471,14 @@ static struct ctl_table fs_table[] = {
.procname = "inode-nr",
.data = &inodes_stat,
.maxlen = 2*sizeof(int),
.maxlen = 2*sizeof(long),
.mode = 0444,
.proc_handler = proc_nr_inodes,
.procname = "inode-state",
.data = &inodes_stat,
.maxlen = 7*sizeof(int),
.maxlen = 7*sizeof(long),
.mode = 0444,
.proc_handler = proc_nr_inodes,
......@@ -1508,7 +1508,7 @@ static struct ctl_table fs_table[] = {
.procname = "dentry-state",
.data = &dentry_stat,
.maxlen = 6*sizeof(int),
.maxlen = 6*sizeof(long),
.mode = 0444,
.proc_handler = proc_nr_dentry,
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment