KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
- state. This holds the regulator output state.
+ Some regulator directories will contain a field called
+ state. This reports the regulator enable status, for
+ regulators which can report that value.
This will be one of the following strings:
'disabled' means the regulator output is OFF and is not
supplying power to the system..
- 'unknown' means software cannot determine the state.
+ 'unknown' means software cannot determine the state, or
+ the reported state is invalid.
NOTE: this field can be used in conjunction with microvolts
and microamps to determine regulator output levels.
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
microvolts. This holds the regulator output voltage setting
- measured in microvolts (i.e. E-6 Volts).
+ measured in microvolts (i.e. E-6 Volts), for regulators
+ which can report that voltage.
NOTE: This value should not be used to determine the regulator
output voltage level as this value is the same regardless of
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
microamps. This holds the regulator output current limit
- setting measured in microamps (i.e. E-6 Amps).
+ setting measured in microamps (i.e. E-6 Amps), for regulators
+ which can report that current.
NOTE: This value should not be used to determine the regulator
output current level as this value is the same regardless of
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
- opmode. This holds the regulator operating mode setting.
+ Some regulator directories will contain a field called
+ opmode. This holds the current regulator operating mode,
+ for regulators which can report it.
The opmode value can be one of the following strings:
'standby'
'unknown'
- The modes are described in include/linux/regulator/regulator.h
+ The modes are described in include/linux/regulator/consumer.h
NOTE: This value should not be used to determine the regulator
output operating mode as this value is the same regardless of
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
min_microvolts. This holds the minimum safe working regulator
- output voltage setting for this domain measured in microvolts.
+ output voltage setting for this domain measured in microvolts,
+ for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microvolts constraint defined by
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
max_microvolts. This holds the maximum safe working regulator
- output voltage setting for this domain measured in microvolts.
+ output voltage setting for this domain measured in microvolts,
+ for regulators which support voltage constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microvolts constraint defined by
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
min_microamps. This holds the minimum safe working regulator
output current limit setting for this domain measured in
- microamps.
+ microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no min microamps constraint defined by
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
max_microamps. This holds the maximum safe working regulator
output current limit setting for this domain measured in
- microamps.
+ microamps, for regulators which support current constraints.
NOTE: this will return the string 'constraint not defined' if
the power domain has no max microamps constraint defined by
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
requested_microamps. This holds the total requested load
current in microamps for this regulator from all its consumer
devices.
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to memory.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to memory voltage defined by
- platform code.
+ the system is suspended to memory, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_disk_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to disk.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to disk voltage defined by
- platform code.
+ the system is suspended to disk, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_standby_microvolts
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_microvolts. This holds the regulator output
voltage setting for this domain measured in microvolts when
- the system is suspended to standby.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to standby voltage defined by
- platform code.
+ the system is suspended to standby, for voltage regulators
+ implementing suspend voltage configuration constraints.
What: /sys/class/regulator/.../suspend_mem_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
- memory.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to memory mode defined by
- platform code.
+ memory, for regulators implementing suspend mode
+ configuration constraints.
What: /sys/class/regulator/.../suspend_disk_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_mode. This holds the regulator operating mode
- setting for this domain when the system is suspended to disk.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to disk mode defined by
- platform code.
+ setting for this domain when the system is suspended to disk,
+ for regulators implementing suspend mode configuration
+ constraints.
What: /sys/class/regulator/.../suspend_standby_mode
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_mode. This holds the regulator operating mode
setting for this domain when the system is suspended to
- standby.
-
- NOTE: this will return the string 'not defined' if
- the power domain has no suspend to standby mode defined by
- platform code.
+ standby, for regulators implementing suspend mode
+ configuration constraints.
What: /sys/class/regulator/.../suspend_mem_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_mem_state. This holds the regulator operating state
- when suspended to memory.
-
- This will be one of the following strings:
+ when suspended to memory, for regulators implementing suspend
+ configuration constraints.
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
What: /sys/class/regulator/.../suspend_disk_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_disk_state. This holds the regulator operating state
- when suspended to disk.
-
- This will be one of the following strings:
+ when suspended to disk, for regulators implementing
+ suspend configuration constraints.
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
What: /sys/class/regulator/.../suspend_standby_state
Date: May 2008
KernelVersion: 2.6.26
Contact: Liam Girdwood <lrg@slimlogic.co.uk>
Description:
- Each regulator directory will contain a field called
+ Some regulator directories will contain a field called
suspend_standby_state. This holds the regulator operating
- state when suspended to standby.
-
- This will be one of the following strings:
+ state when suspended to standby, for regulators implementing
+ suspend configuration constraints.
- 'enabled'
- 'disabled'
- 'not defined'
+ This will be one of the same strings reported by
+ the "state" attribute.
kernel-api.xml filesystems.xml lsm.xml usb.xml kgdb.xml \
gadget.xml libata.xml mtdnand.xml librs.xml rapidio.xml \
genericirq.xml s390-drivers.xml uio-howto.xml scsi.xml \
- mac80211.xml debugobjects.xml sh.xml
+ mac80211.xml debugobjects.xml sh.xml regulator.xml
###
# The build process is as follows (targets):
--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" []>
+
+<book id="regulator-api">
+ <bookinfo>
+ <title>Voltage and current regulator API</title>
+
+ <authorgroup>
+ <author>
+ <firstname>Liam</firstname>
+ <surname>Girdwood</surname>
+ <affiliation>
+ <address>
+ <email>lrg@slimlogic.co.uk</email>
+ </address>
+ </affiliation>
+ </author>
+ <author>
+ <firstname>Mark</firstname>
+ <surname>Brown</surname>
+ <affiliation>
+ <orgname>Wolfson Microelectronics</orgname>
+ <address>
+ <email>broonie@opensource.wolfsonmicro.com</email>
+ </address>
+ </affiliation>
+ </author>
+ </authorgroup>
+
+ <copyright>
+ <year>2007-2008</year>
+ <holder>Wolfson Microelectronics</holder>
+ </copyright>
+ <copyright>
+ <year>2008</year>
+ <holder>Liam Girdwood</holder>
+ </copyright>
+
+ <legalnotice>
+ <para>
+ This documentation is free software; you can redistribute
+ it and/or modify it under the terms of the GNU General Public
+ License version 2 as published by the Free Software Foundation.
+ </para>
+
+ <para>
+ This program is distributed in the hope that it will be
+ useful, but WITHOUT ANY WARRANTY; without even the implied
+ warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ See the GNU General Public License for more details.
+ </para>
+
+ <para>
+ You should have received a copy of the GNU General Public
+ License along with this program; if not, write to the Free
+ Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+ MA 02111-1307 USA
+ </para>
+
+ <para>
+ For more details see the file COPYING in the source
+ distribution of Linux.
+ </para>
+ </legalnotice>
+ </bookinfo>
+
+<toc></toc>
+
+ <chapter id="intro">
+ <title>Introduction</title>
+ <para>
+ This framework is designed to provide a standard kernel
+ interface to control voltage and current regulators.
+ </para>
+ <para>
+ The intention is to allow systems to dynamically control
+ regulator power output in order to save power and prolong
+ battery life. This applies to both voltage regulators (where
+ voltage output is controllable) and current sinks (where current
+ limit is controllable).
+ </para>
+ <para>
+ Note that additional (and currently more complete) documentation
+ is available in the Linux kernel source under
+ <filename>Documentation/power/regulator</filename>.
+ </para>
+
+ <sect1 id="glossary">
+ <title>Glossary</title>
+ <para>
+ The regulator API uses a number of terms which may not be
+ familiar:
+ </para>
+ <glossary>
+
+ <glossentry>
+ <glossterm>Regulator</glossterm>
+ <glossdef>
+ <para>
+ Electronic device that supplies power to other devices. Most
+ regulators can enable and disable their output and some can also
+ control their output voltage or current.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Consumer</glossterm>
+ <glossdef>
+ <para>
+ Electronic device which consumes power provided by a regulator.
+ These may either be static, requiring only a fixed supply, or
+ dynamic, requiring active management of the regulator at
+ runtime.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Power Domain</glossterm>
+ <glossdef>
+ <para>
+ The electronic circuit supplied by a given regulator, including
+ the regulator and all consumer devices. The configuration of
+ the regulator is shared between all the components in the
+ circuit.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry>
+ <glossterm>Power Management Integrated Circuit</glossterm>
+ <acronym>PMIC</acronym>
+ <glossdef>
+ <para>
+ An IC which contains numerous regulators and often also other
+ subsystems. In an embedded system the primary PMIC is often
+ equivalent to a combination of the PSU and southbridge in a
+ desktop system.
+ </para>
+ </glossdef>
+ </glossentry>
+ </glossary>
+ </sect1>
+ </chapter>
+
+ <chapter id="consumer">
+ <title>Consumer driver interface</title>
+ <para>
+ This offers a similar API to the kernel clock framework.
+ Consumer drivers use <link
+ linkend='API-regulator-get'>get</link> and <link
+ linkend='API-regulator-put'>put</link> operations to acquire and
+ release regulators. Functions are
+ provided to <link linkend='API-regulator-enable'>enable</link>
+ and <link linkend='API-regulator-disable'>disable</link> the
+ reguator and to get and set the runtime parameters of the
+ regulator.
+ </para>
+ <para>
+ When requesting regulators consumers use symbolic names for their
+ supplies, such as "Vcc", which are mapped into actual regulator
+ devices by the machine interface.
+ </para>
+ <para>
+ A stub version of this API is provided when the regulator
+ framework is not in use in order to minimise the need to use
+ ifdefs.
+ </para>
+
+ <sect1 id="consumer-enable">
+ <title>Enabling and disabling</title>
+ <para>
+ The regulator API provides reference counted enabling and
+ disabling of regulators. Consumer devices use the <function><link
+ linkend='API-regulator-enable'>regulator_enable</link></function>
+ and <function><link
+ linkend='API-regulator-disable'>regulator_disable</link>
+ </function> functions to enable and disable regulators. Calls
+ to the two functions must be balanced.
+ </para>
+ <para>
+ Note that since multiple consumers may be using a regulator and
+ machine constraints may not allow the regulator to be disabled
+ there is no guarantee that calling
+ <function>regulator_disable</function> will actually cause the
+ supply provided by the regulator to be disabled. Consumer
+ drivers should assume that the regulator may be enabled at all
+ times.
+ </para>
+ </sect1>
+
+ <sect1 id="consumer-config">
+ <title>Configuration</title>
+ <para>
+ Some consumer devices may need to be able to dynamically
+ configure their supplies. For example, MMC drivers may need to
+ select the correct operating voltage for their cards. This may
+ be done while the regulator is enabled or disabled.
+ </para>
+ <para>
+ The <function><link
+ linkend='API-regulator-set-voltage'>regulator_set_voltage</link>
+ </function> and <function><link
+ linkend='API-regulator-set-current-limit'
+ >regulator_set_current_limit</link>
+ </function> functions provide the primary interface for this.
+ Both take ranges of voltages and currents, supporting drivers
+ that do not require a specific value (eg, CPU frequency scaling
+ normally permits the CPU to use a wider range of supply
+ voltages at lower frequencies but does not require that the
+ supply voltage be lowered). Where an exact value is required
+ both minimum and maximum values should be identical.
+ </para>
+ </sect1>
+
+ <sect1 id="consumer-callback">
+ <title>Callbacks</title>
+ <para>
+ Callbacks may also be <link
+ linkend='API-regulator-register-notifier'>registered</link>
+ for events such as regulation failures.
+ </para>
+ </sect1>
+ </chapter>
+
+ <chapter id="driver">
+ <title>Regulator driver interface</title>
+ <para>
+ Drivers for regulator chips <link
+ linkend='API-regulator-register'>register</link> the regulators
+ with the regulator core, providing operations structures to the
+ core. A <link
+ linkend='API-regulator-notifier-call-chain'>notifier</link> interface
+ allows error conditions to be reported to the core.
+ </para>
+ <para>
+ Registration should be triggered by explicit setup done by the
+ platform, supplying a <link
+ linkend='API-struct-regulator-init-data'>struct
+ regulator_init_data</link> for the regulator containing
+ <link linkend='machine-constraint'>constraint</link> and
+ <link linkend='machine-supply'>supply</link> information.
+ </para>
+ </chapter>
+
+ <chapter id="machine">
+ <title>Machine interface</title>
+ <para>
+ This interface provides a way to define how regulators are
+ connected to consumers on a given system and what the valid
+ operating parameters are for the system.
+ </para>
+
+ <sect1 id="machine-supply">
+ <title>Supplies</title>
+ <para>
+ Regulator supplies are specified using <link
+ linkend='API-struct-regulator-consumer-supply'>struct
+ regulator_consumer_supply</link>. This is done at
+ <link linkend='driver'>driver registration
+ time</link> as part of the machine constraints.
+ </para>
+ </sect1>
+
+ <sect1 id="machine-constraint">
+ <title>Constraints</title>
+ <para>
+ As well as definining the connections the machine interface
+ also provides constraints definining the operations that
+ clients are allowed to perform and the parameters that may be
+ set. This is required since generally regulator devices will
+ offer more flexibility than it is safe to use on a given
+ system, for example supporting higher supply voltages than the
+ consumers are rated for.
+ </para>
+ <para>
+ This is done at <link linkend='driver'>driver
+ registration time</link> by providing a <link
+ linkend='API-struct-regulation-constraints'>struct
+ regulation_constraints</link>.
+ </para>
+ <para>
+ The constraints may also specify an initial configuration for the
+ regulator in the constraints, which is particularly useful for
+ use with static consumers.
+ </para>
+ </sect1>
+ </chapter>
+
+ <chapter id="api">
+ <title>API reference</title>
+ <para>
+ Due to limitations of the kernel documentation framework and the
+ existing layout of the source code the entire regulator API is
+ documented here.
+ </para>
+!Iinclude/linux/regulator/consumer.h
+!Iinclude/linux/regulator/machine.h
+!Iinclude/linux/regulator/driver.h
+!Edrivers/regulator/core.c
+ </chapter>
+</book>
- Reference-count design for elements of lists/arrays protected by RCU
rcu.txt
- RCU Concepts
+rcubarrier.txt
+ - Unloading modules that use RCU callbacks
RTFP.txt
- List of RCU papers (bibliography) going back to 1980.
torture.txt
--- /dev/null
+RCU and Unloadable Modules
+
+[Originally published in LWN Jan. 14, 2007: http://lwn.net/Articles/217484/]
+
+RCU (read-copy update) is a synchronization mechanism that can be thought
+of as a replacement for read-writer locking (among other things), but with
+very low-overhead readers that are immune to deadlock, priority inversion,
+and unbounded latency. RCU read-side critical sections are delimited
+by rcu_read_lock() and rcu_read_unlock(), which, in non-CONFIG_PREEMPT
+kernels, generate no code whatsoever.
+
+This means that RCU writers are unaware of the presence of concurrent
+readers, so that RCU updates to shared data must be undertaken quite
+carefully, leaving an old version of the data structure in place until all
+pre-existing readers have finished. These old versions are needed because
+such readers might hold a reference to them. RCU updates can therefore be
+rather expensive, and RCU is thus best suited for read-mostly situations.
+
+How can an RCU writer possibly determine when all readers are finished,
+given that readers might well leave absolutely no trace of their
+presence? There is a synchronize_rcu() primitive that blocks until all
+pre-existing readers have completed. An updater wishing to delete an
+element p from a linked list might do the following, while holding an
+appropriate lock, of course:
+
+ list_del_rcu(p);
+ synchronize_rcu();
+ kfree(p);
+
+But the above code cannot be used in IRQ context -- the call_rcu()
+primitive must be used instead. This primitive takes a pointer to an
+rcu_head struct placed within the RCU-protected data structure and
+another pointer to a function that may be invoked later to free that
+structure. Code to delete an element p from the linked list from IRQ
+context might then be as follows:
+
+ list_del_rcu(p);
+ call_rcu(&p->rcu, p_callback);
+
+Since call_rcu() never blocks, this code can safely be used from within
+IRQ context. The function p_callback() might be defined as follows:
+
+ static void p_callback(struct rcu_head *rp)
+ {
+ struct pstruct *p = container_of(rp, struct pstruct, rcu);
+
+ kfree(p);
+ }
+
+
+Unloading Modules That Use call_rcu()
+
+But what if p_callback is defined in an unloadable module?
+
+If we unload the module while some RCU callbacks are pending,
+the CPUs executing these callbacks are going to be severely
+disappointed when they are later invoked, as fancifully depicted at
+http://lwn.net/images/ns/kernel/rcu-drop.jpg.
+
+We could try placing a synchronize_rcu() in the module-exit code path,
+but this is not sufficient. Although synchronize_rcu() does wait for a
+grace period to elapse, it does not wait for the callbacks to complete.
+
+One might be tempted to try several back-to-back synchronize_rcu()
+calls, but this is still not guaranteed to work. If there is a very
+heavy RCU-callback load, then some of the callbacks might be deferred
+in order to allow other processing to proceed. Such deferral is required
+in realtime kernels in order to avoid excessive scheduling latencies.
+
+
+rcu_barrier()
+
+We instead need the rcu_barrier() primitive. This primitive is similar
+to synchronize_rcu(), but instead of waiting solely for a grace
+period to elapse, it also waits for all outstanding RCU callbacks to
+complete. Pseudo-code using rcu_barrier() is as follows:
+
+ 1. Prevent any new RCU callbacks from being posted.
+ 2. Execute rcu_barrier().
+ 3. Allow the module to be unloaded.
+
+Quick Quiz #1: Why is there no srcu_barrier()?
+
+The rcutorture module makes use of rcu_barrier in its exit function
+as follows:
+
+ 1 static void
+ 2 rcu_torture_cleanup(void)
+ 3 {
+ 4 int i;
+ 5
+ 6 fullstop = 1;
+ 7 if (shuffler_task != NULL) {
+ 8 VERBOSE_PRINTK_STRING("Stopping rcu_torture_shuffle task");
+ 9 kthread_stop(shuffler_task);
+10 }
+11 shuffler_task = NULL;
+12
+13 if (writer_task != NULL) {
+14 VERBOSE_PRINTK_STRING("Stopping rcu_torture_writer task");
+15 kthread_stop(writer_task);
+16 }
+17 writer_task = NULL;
+18
+19 if (reader_tasks != NULL) {
+20 for (i = 0; i < nrealreaders; i++) {
+21 if (reader_tasks[i] != NULL) {
+22 VERBOSE_PRINTK_STRING(
+23 "Stopping rcu_torture_reader task");
+24 kthread_stop(reader_tasks[i]);
+25 }
+26 reader_tasks[i] = NULL;
+27 }
+28 kfree(reader_tasks);
+29 reader_tasks = NULL;
+30 }
+31 rcu_torture_current = NULL;
+32
+33 if (fakewriter_tasks != NULL) {
+34 for (i = 0; i < nfakewriters; i++) {
+35 if (fakewriter_tasks[i] != NULL) {
+36 VERBOSE_PRINTK_STRING(
+37 "Stopping rcu_torture_fakewriter task");
+38 kthread_stop(fakewriter_tasks[i]);
+39 }
+40 fakewriter_tasks[i] = NULL;
+41 }
+42 kfree(fakewriter_tasks);
+43 fakewriter_tasks = NULL;
+44 }
+45
+46 if (stats_task != NULL) {
+47 VERBOSE_PRINTK_STRING("Stopping rcu_torture_stats task");
+48 kthread_stop(stats_task);
+49 }
+50 stats_task = NULL;
+51
+52 /* Wait for all RCU callbacks to fire. */
+53 rcu_barrier();
+54
+55 rcu_torture_stats_print(); /* -After- the stats thread is stopped! */
+56
+57 if (cur_ops->cleanup != NULL)
+58 cur_ops->cleanup();
+59 if (atomic_read(&n_rcu_torture_error))
+60 rcu_torture_print_module_parms("End of test: FAILURE");
+61 else
+62 rcu_torture_print_module_parms("End of test: SUCCESS");
+63 }
+
+Line 6 sets a global variable that prevents any RCU callbacks from
+re-posting themselves. This will not be necessary in most cases, since
+RCU callbacks rarely include calls to call_rcu(). However, the rcutorture
+module is an exception to this rule, and therefore needs to set this
+global variable.
+
+Lines 7-50 stop all the kernel tasks associated with the rcutorture
+module. Therefore, once execution reaches line 53, no more rcutorture
+RCU callbacks will be posted. The rcu_barrier() call on line 53 waits
+for any pre-existing callbacks to complete.
+
+Then lines 55-62 print status and do operation-specific cleanup, and
+then return, permitting the module-unload operation to be completed.
+
+Quick Quiz #2: Is there any other situation where rcu_barrier() might
+ be required?
+
+Your module might have additional complications. For example, if your
+module invokes call_rcu() from timers, you will need to first cancel all
+the timers, and only then invoke rcu_barrier() to wait for any remaining
+RCU callbacks to complete.
+
+
+Implementing rcu_barrier()
+
+Dipankar Sarma's implementation of rcu_barrier() makes use of the fact
+that RCU callbacks are never reordered once queued on one of the per-CPU
+queues. His implementation queues an RCU callback on each of the per-CPU
+callback queues, and then waits until they have all started executing, at
+which point, all earlier RCU callbacks are guaranteed to have completed.
+
+The original code for rcu_barrier() was as follows:
+
+ 1 void rcu_barrier(void)
+ 2 {
+ 3 BUG_ON(in_interrupt());
+ 4 /* Take cpucontrol mutex to protect against CPU hotplug */
+ 5 mutex_lock(&rcu_barrier_mutex);
+ 6 init_completion(&rcu_barrier_completion);
+ 7 atomic_set(&rcu_barrier_cpu_count, 0);
+ 8 on_each_cpu(rcu_barrier_func, NULL, 0, 1);
+ 9 wait_for_completion(&rcu_barrier_completion);
+10 mutex_unlock(&rcu_barrier_mutex);
+11 }
+
+Line 3 verifies that the caller is in process context, and lines 5 and 10
+use rcu_barrier_mutex to ensure that only one rcu_barrier() is using the
+global completion and counters at a time, which are initialized on lines
+6 and 7. Line 8 causes each CPU to invoke rcu_barrier_func(), which is
+shown below. Note that the final "1" in on_each_cpu()'s argument list
+ensures that all the calls to rcu_barrier_func() will have completed
+before on_each_cpu() returns. Line 9 then waits for the completion.
+
+This code was rewritten in 2008 to support rcu_barrier_bh() and
+rcu_barrier_sched() in addition to the original rcu_barrier().
+
+The rcu_barrier_func() runs on each CPU, where it invokes call_rcu()
+to post an RCU callback, as follows:
+
+ 1 static void rcu_barrier_func(void *notused)
+ 2 {
+ 3 int cpu = smp_processor_id();
+ 4 struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+ 5 struct rcu_head *head;
+ 6
+ 7 head = &rdp->barrier;
+ 8 atomic_inc(&rcu_barrier_cpu_count);
+ 9 call_rcu(head, rcu_barrier_callback);
+10 }
+
+Lines 3 and 4 locate RCU's internal per-CPU rcu_data structure,
+which contains the struct rcu_head that needed for the later call to
+call_rcu(). Line 7 picks up a pointer to this struct rcu_head, and line
+8 increments a global counter. This counter will later be decremented
+by the callback. Line 9 then registers the rcu_barrier_callback() on
+the current CPU's queue.
+
+The rcu_barrier_callback() function simply atomically decrements the
+rcu_barrier_cpu_count variable and finalizes the completion when it
+reaches zero, as follows:
+
+ 1 static void rcu_barrier_callback(struct rcu_head *notused)
+ 2 {
+ 3 if (atomic_dec_and_test(&rcu_barrier_cpu_count))
+ 4 complete(&rcu_barrier_completion);
+ 5 }
+
+Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
+ immediately (thus incrementing rcu_barrier_cpu_count to the
+ value one), but the other CPU's rcu_barrier_func() invocations
+ are delayed for a full grace period? Couldn't this result in
+ rcu_barrier() returning prematurely?
+
+
+rcu_barrier() Summary
+
+The rcu_barrier() primitive has seen relatively little use, since most
+code using RCU is in the core kernel rather than in modules. However, if
+you are using RCU from an unloadable module, you need to use rcu_barrier()
+so that your module may be safely unloaded.
+
+
+Answers to Quick Quizzes
+
+Quick Quiz #1: Why is there no srcu_barrier()?
+
+Answer: Since there is no call_srcu(), there can be no outstanding SRCU
+ callbacks. Therefore, there is no need to wait for them.
+
+Quick Quiz #2: Is there any other situation where rcu_barrier() might
+ be required?
+
+Answer: Interestingly enough, rcu_barrier() was not originally
+ implemented for module unloading. Nikita Danilov was using
+ RCU in a filesystem, which resulted in a similar situation at
+ filesystem-unmount time. Dipankar Sarma coded up rcu_barrier()
+ in response, so that Nikita could invoke it during the
+ filesystem-unmount process.
+
+ Much later, yours truly hit the RCU module-unload problem when
+ implementing rcutorture, and found that rcu_barrier() solves
+ this problem as well.
+
+Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes
+ immediately (thus incrementing rcu_barrier_cpu_count to the
+ value one), but the other CPU's rcu_barrier_func() invocations
+ are delayed for a full grace period? Couldn't this result in
+ rcu_barrier() returning prematurely?
+
+Answer: This cannot happen. The reason is that on_each_cpu() has its last
+ argument, the wait flag, set to "1". This flag is passed through
+ to smp_call_function() and further to smp_call_function_on_cpu(),
+ causing this latter to spin until the cross-CPU invocation of
+ rcu_barrier_func() has completed. This by itself would prevent
+ a grace period from completing on non-CONFIG_PREEMPT kernels,
+ since each CPU must undergo a context switch (or other quiescent
+ state) before the grace period can complete. However, this is
+ of no use in CONFIG_PREEMPT kernels.
+
+ Therefore, on_each_cpu() disables preemption across its call
+ to smp_call_function() and also across the local call to
+ rcu_barrier_func(). This prevents the local CPU from context
+ switching, again preventing grace periods from completing. This
+ means that all CPUs have executed rcu_barrier_func() before
+ the first rcu_barrier_callback() can possibly execute, in turn
+ preventing rcu_barrier_cpu_count from prematurely reaching zero.
+
+ Currently, -rt implementations of RCU keep but a single global
+ queue for RCU callbacks, and thus do not suffer from this
+ problem. However, when the -rt RCU eventually does have per-CPU
+ callback queues, things will have to change. One simple change
+ is to add an rcu_read_lock() before line 8 of rcu_barrier()
+ and an rcu_read_unlock() after line 8 of this same function. If
+ you can think of a better change, please let me know!
--- /dev/null
+March 2008
+Jan-Simon Moeller, dl9pf@gmx.de
+
+
+How to deal with bad memory e.g. reported by memtest86+ ?
+#########################################################
+
+There are three possibilities I know of:
+
+1) Reinsert/swap the memory modules
+
+2) Buy new modules (best!) or try to exchange the memory
+ if you have spare-parts
+
+3) Use BadRAM or memmap
+
+This Howto is about number 3) .
+
+
+BadRAM
+######
+BadRAM is the actively developed and available as kernel-patch
+here: http://rick.vanrein.org/linux/badram/
+
+For more details see the BadRAM documentation.
+
+memmap
+######
+
+memmap is already in the kernel and usable as kernel-parameter at
+boot-time. Its syntax is slightly strange and you may need to
+calculate the values by yourself!
+
+Syntax to exclude a memory area (see kernel-parameters.txt for details):
+memmap=<size>$<address>
+
+Example: memtest86+ reported here errors at address 0x18691458, 0x18698424 and
+ some others. All had 0x1869xxxx in common, so I chose a pattern of
+ 0x18690000,0xffff0000.
+
+With the numbers of the example above:
+memmap=64K$0x18690000
+ or
+memmap=0x10000$0x18690000
+
containing the following files describing that cgroup:
- tasks: list of tasks (by pid) attached to that cgroup
- - releasable flag: cgroup currently removeable?
- notify_on_release flag: run the release agent on exit?
- release_agent: the path to use for release notifications (this file
exists in the top cgroup only)
In this directory you can find several files:
# ls
-notify_on_release releasable tasks
+notify_on_release tasks
(plus whatever files added by the attached subsystems)
Now attach your shell to this cgroup:
create() method has been called for the new cgroup).
void pre_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
-(cgroup_mutex held by caller)
Called before checking the reference count on each subsystem. This may
be useful for subsystems which have some extra references even if
void attach(struct cgroup_subsys *ss, struct cgroup *cgrp,
struct cgroup *old_cgrp, struct task_struct *task)
+(cgroup_mutex held by caller)
Called after the task has been attached to the cgroup, to allow any
post-attachment activity that requires memory allocations or blocking.
Called during task exit.
int populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
+(cgroup_mutex held by caller)
Called after creation of a cgroup to allow a subsystem to populate
the cgroup directory with file entries. The subsystem should make
always handled well.
void post_clone(struct cgroup_subsys *ss, struct cgroup *cgrp)
+(cgroup_mutex held by caller)
Called at the end of cgroup_clone() to do any paramater
initialization which might be required before a task could attach. For
up.
void bind(struct cgroup_subsys *ss, struct cgroup *root)
-(cgroup_mutex held by caller)
+(cgroup_mutex and ss->hierarchy_mutex held by caller)
Called when a cgroup subsystem is rebound to a different hierarchy
and root cgroup. Currently this will only involve movement between
--- /dev/null
+Memory Resource Controller(Memcg) Implementation Memo.
+Last Updated: 2008/12/15
+Base Kernel Version: based on 2.6.28-rc8-mm.
+
+Because VM is getting complex (one of reasons is memcg...), memcg's behavior
+is complex. This is a document for memcg's internal behavior.
+Please note that implementation details can be changed.
+
+(*) Topics on API should be in Documentation/controllers/memory.txt)
+
+0. How to record usage ?
+ 2 objects are used.
+
+ page_cgroup ....an object per page.
+ Allocated at boot or memory hotplug. Freed at memory hot removal.
+
+ swap_cgroup ... an entry per swp_entry.
+ Allocated at swapon(). Freed at swapoff().
+
+ The page_cgroup has USED bit and double count against a page_cgroup never
+ occurs. swap_cgroup is used only when a charged page is swapped-out.
+
+1. Charge
+
+ a page/swp_entry may be charged (usage += PAGE_SIZE) at
+
+ mem_cgroup_newpage_charge()
+ Called at new page fault and Copy-On-Write.
+
+ mem_cgroup_try_charge_swapin()
+ Called at do_swap_page() (page fault on swap entry) and swapoff.
+ Followed by charge-commit-cancel protocol. (With swap accounting)
+ At commit, a charge recorded in swap_cgroup is removed.
+
+ mem_cgroup_cache_charge()
+ Called at add_to_page_cache()
+
+ mem_cgroup_cache_charge_swapin()
+ Called at shmem's swapin.
+
+ mem_cgroup_prepare_migration()
+ Called before migration. "extra" charge is done and followed by
+ charge-commit-cancel protocol.
+ At commit, charge against oldpage or newpage will be committed.
+
+2. Uncharge
+ a page/swp_entry may be uncharged (usage -= PAGE_SIZE) by
+
+ mem_cgroup_uncharge_page()
+ Called when an anonymous page is fully unmapped. I.e., mapcount goes
+ to 0. If the page is SwapCache, uncharge is delayed until
+ mem_cgroup_uncharge_swapcache().
+
+ mem_cgroup_uncharge_cache_page()
+ Called when a page-cache is deleted from radix-tree. If the page is
+ SwapCache, uncharge is delayed until mem_cgroup_uncharge_swapcache().
+
+ mem_cgroup_uncharge_swapcache()
+ Called when SwapCache is removed from radix-tree. The charge itself
+ is moved to swap_cgroup. (If mem+swap controller is disabled, no
+ charge to swap occurs.)
+
+ mem_cgroup_uncharge_swap()
+ Called when swp_entry's refcnt goes down to 0. A charge against swap
+ disappears.
+
+ mem_cgroup_end_migration(old, new)
+ At success of migration old is uncharged (if necessary), a charge
+ to new page is committed. At failure, charge to old page is committed.
+
+3. charge-commit-cancel
+ In some case, we can't know this "charge" is valid or not at charging
+ (because of races).
+ To handle such case, there are charge-commit-cancel functions.
+ mem_cgroup_try_charge_XXX
+ mem_cgroup_commit_charge_XXX
+ mem_cgroup_cancel_charge_XXX
+ these are used in swap-in and migration.
+
+ At try_charge(), there are no flags to say "this page is charged".
+ at this point, usage += PAGE_SIZE.
+
+ At commit(), the function checks the page should be charged or not
+ and set flags or avoid charging.(usage -= PAGE_SIZE)
+
+ At cancel(), simply usage -= PAGE_SIZE.
+
+Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y.
+
+4. Anonymous
+ Anonymous page is newly allocated at
+ - page fault into MAP_ANONYMOUS mapping.
+ - Copy-On-Write.
+ It is charged right after it's allocated before doing any page table
+ related operations. Of course, it's uncharged when another page is used
+ for the fault address.
+
+ At freeing anonymous page (by exit() or munmap()), zap_pte() is called
+ and pages for ptes are freed one by one.(see mm/memory.c). Uncharges
+ are done at page_remove_rmap() when page_mapcount() goes down to 0.
+
+ Another page freeing is by page-reclaim (vmscan.c) and anonymous
+ pages are swapped out. In this case, the page is marked as
+ PageSwapCache(). uncharge() routine doesn't uncharge the page marked
+ as SwapCache(). It's delayed until __delete_from_swap_cache().
+
+ 4.1 Swap-in.
+ At swap-in, the page is taken from swap-cache. There are 2 cases.
+
+ (a) If the SwapCache is newly allocated and read, it has no charges.
+ (b) If the SwapCache has been mapped by processes, it has been
+ charged already.
+
+ This swap-in is one of the most complicated work. In do_swap_page(),
+ following events occur when pte is unchanged.
+
+ (1) the page (SwapCache) is looked up.
+ (2) lock_page()
+ (3) try_charge_swapin()
+ (4) reuse_swap_page() (may call delete_swap_cache())
+ (5) commit_charge_swapin()
+ (6) swap_free().
+
+ Considering following situation for example.
+
+ (A) The page has not been charged before (2) and reuse_swap_page()
+ doesn't call delete_from_swap_cache().
+ (B) The page has not been charged before (2) and reuse_swap_page()
+ calls delete_from_swap_cache().
+ (C) The page has been charged before (2) and reuse_swap_page() doesn't
+ call delete_from_swap_cache().
+ (D) The page has been charged before (2) and reuse_swap_page() calls
+ delete_from_swap_cache().
+
+ memory.usage/memsw.usage changes to this page/swp_entry will be
+ Case (A) (B) (C) (D)
+ Event
+ Before (2) 0/ 1 0/ 1 1/ 1 1/ 1
+ ===========================================
+ (3) +1/+1 +1/+1 +1/+1 +1/+1
+ (4) - 0/ 0 - -1/ 0
+ (5) 0/-1 0/ 0 -1/-1 0/ 0
+ (6) - 0/-1 - 0/-1
+ ===========================================
+ Result 1/ 1 1/ 1 1/ 1 1/ 1
+
+ In any cases, charges to this page should be 1/ 1.
+
+ 4.2 Swap-out.
+ At swap-out, typical state transition is below.
+
+ (a) add to swap cache. (marked as SwapCache)
+ swp_entry's refcnt += 1.
+ (b) fully unmapped.
+ swp_entry's refcnt += # of ptes.
+ (c) write back to swap.
+ (d) delete from swap cache. (remove from SwapCache)
+ swp_entry's refcnt -= 1.
+
+
+ At (b), the page is marked as SwapCache and not uncharged.
+ At (d), the page is removed from SwapCache and a charge in page_cgroup
+ is moved to swap_cgroup.
+
+ Finally, at task exit,
+ (e) zap_pte() is called and swp_entry's refcnt -=1 -> 0.
+ Here, a charge in swap_cgroup disappears.
+
+5. Page Cache
+ Page Cache is charged at
+ - add_to_page_cache_locked().
+
+ uncharged at
+ - __remove_from_page_cache().
+
+ The logic is very clear. (About migration, see below)
+ Note: __remove_from_page_cache() is called by remove_from_page_cache()
+ and __remove_mapping().
+
+6. Shmem(tmpfs) Page Cache
+ Memcg's charge/uncharge have special handlers of shmem. The best way
+ to understand shmem's page state transition is to read mm/shmem.c.
+ But brief explanation of the behavior of memcg around shmem will be
+ helpful to understand the logic.
+
+ Shmem's page (just leaf page, not direct/indirect block) can be on
+ - radix-tree of shmem's inode.
+ - SwapCache.
+ - Both on radix-tree and SwapCache. This happens at swap-in
+ and swap-out,
+
+ It's charged when...
+ - A new page is added to shmem's radix-tree.
+ - A swp page is read. (move a charge from swap_cgroup to page_cgroup)
+ It's uncharged when
+ - A page is removed from radix-tree and not SwapCache.
+ - When SwapCache is removed, a charge is moved to swap_cgroup.
+ - When swp_entry's refcnt goes down to 0, a charge in swap_cgroup
+ disappears.
+
+7. Page Migration
+ One of the most complicated functions is page-migration-handler.
+ Memcg has 2 routines. Assume that we are migrating a page's contents
+ from OLDPAGE to NEWPAGE.
+
+ Usual migration logic is..
+ (a) remove the page from LRU.
+ (b) allocate NEWPAGE (migration target)
+ (c) lock by lock_page().
+ (d) unmap all mappings.
+ (e-1) If necessary, replace entry in radix-tree.
+ (e-2) move contents of a page.
+ (f) map all mappings again.
+ (g) pushback the page to LRU.
+ (-) OLDPAGE will be freed.
+
+ Before (g), memcg should complete all necessary charge/uncharge to
+ NEWPAGE/OLDPAGE.
+
+ The point is....
+ - If OLDPAGE is anonymous, all charges will be dropped at (d) because
+ try_to_unmap() drops all mapcount and the page will not be
+ SwapCache.
+
+ - If OLDPAGE is SwapCache, charges will be kept at (g) because
+ __delete_from_swap_cache() isn't called at (e-1)
+
+ - If OLDPAGE is page-cache, charges will be kept at (g) because
+ remove_from_swap_cache() isn't called at (e-1)
+
+ memcg provides following hooks.
+
+ - mem_cgroup_prepare_migration(OLDPAGE)
+ Called after (b) to account a charge (usage += PAGE_SIZE) against
+ memcg which OLDPAGE belongs to.
+
+ - mem_cgroup_end_migration(OLDPAGE, NEWPAGE)
+ Called after (f) before (g).
+ If OLDPAGE is used, commit OLDPAGE again. If OLDPAGE is already
+ charged, a charge by prepare_migration() is automatically canceled.
+ If NEWPAGE is used, commit NEWPAGE and uncharge OLDPAGE.
+
+ But zap_pte() (by exit or munmap) can be called while migration,
+ we have to check if OLDPAGE/NEWPAGE is a valid page after commit().
+
+8. LRU
+ Each memcg has its own private LRU. Now, it's handling is under global
+ VM's control (means that it's handled under global zone->lru_lock).
+ Almost all routines around memcg's LRU is called by global LRU's
+ list management functions under zone->lru_lock().
+
+ A special function is mem_cgroup_isolate_pages(). This scans
+ memcg's private LRU and call __isolate_lru_page() to extract a page
+ from LRU.
+ (By __isolate_lru_page(), the page is removed from both of global and
+ private LRU.)
+
+
+9. Typical Tests.
+
+ Tests for racy cases.
+
+ 9.1 Small limit to memcg.
+ When you do test to do racy case, it's good test to set memcg's limit
+ to be very small rather than GB. Many races found in the test under
+ xKB or xxMB limits.
+ (Memory behavior under GB and Memory behavior under MB shows very
+ different situation.)
+
+ 9.2 Shmem
+ Historically, memcg's shmem handling was poor and we saw some amount
+ of troubles here. This is because shmem is page-cache but can be
+ SwapCache. Test with shmem/tmpfs is always good test.
+
+ 9.3 Migration
+ For NUMA, migration is an another special case. To do easy test, cpuset
+ is useful. Following is a sample script to do migration.
+
+ mount -t cgroup -o cpuset none /opt/cpuset
+
+ mkdir /opt/cpuset/01
+ echo 1 > /opt/cpuset/01/cpuset.cpus
+ echo 0 > /opt/cpuset/01/cpuset.mems
+ echo 1 > /opt/cpuset/01/cpuset.memory_migrate
+ mkdir /opt/cpuset/02
+ echo 1 > /opt/cpuset/02/cpuset.cpus
+ echo 1 > /opt/cpuset/02/cpuset.mems
+ echo 1 > /opt/cpuset/02/cpuset.memory_migrate
+
+ In above set, when you moves a task from 01 to 02, page migration to
+ node 0 to node 1 will occur. Following is a script to migrate all
+ under cpuset.
+ --
+ move_task()
+ {
+ for pid in $1
+ do
+ /bin/echo $pid >$2/tasks 2>/dev/null
+ echo -n $pid
+ echo -n " "
+ done
+ echo END
+ }
+
+ G1_TASK=`cat ${G1}/tasks`
+ G2_TASK=`cat ${G2}/tasks`
+ move_task "${G1_TASK}" ${G2} &
+ --
+ 9.4 Memory hotplug.
+ memory hotplug test is one of good test.
+ to offline memory, do following.
+ # echo offline > /sys/devices/system/memory/memoryXXX/state
+ (XXX is the place of memory)
+ This is an easy way to test page migration, too.
+
+ 9.5 mkdir/rmdir
+ When using hierarchy, mkdir/rmdir test should be done.
+ Use tests like the following.
+
+ echo 1 >/opt/cgroup/01/memory/use_hierarchy
+ mkdir /opt/cgroup/01/child_a
+ mkdir /opt/cgroup/01/child_b
+
+ set limit to 01.
+ add limit to 01/child_b
+ run jobs under child_a and child_b
+
+ create/delete following groups at random while jobs are running.
+ /opt/cgroup/01/child_a/child_aa
+ /opt/cgroup/01/child_b/child_bb
+ /opt/cgroup/01/child_c
+
+ running new jobs in new group is also good.
+
+ 9.6 Mount with other subsystems.
+ Mounting with other subsystems is a good test because there is a
+ race and lock dependency with other cgroup subsystems.
+
+ example)
+ # mount -t cgroup none /cgroup -t cpuset,memory,cpu,devices
+
+ and do task move, mkdir, rmdir etc...under this.
page will eventually get charged for it (once it is uncharged from
the cgroup that brought it in -- this will happen on memory pressure).
-2.4 Reclaim
+Exception: If CONFIG_CGROUP_CGROUP_MEM_RES_CTLR_SWAP is not used..
+When you do swapoff and make swapped-out pages of shmem(tmpfs) to
+be backed into memory in force, charges for pages are accounted against the
+caller of swapoff rather than the users of shmem.
+
+
+2.4 Swap Extension (CONFIG_CGROUP_MEM_RES_CTLR_SWAP)
+Swap Extension allows you to record charge for swap. A swapped-in page is
+charged back to original page allocator if possible.
+
+When swap is accounted, following files are added.
+ - memory.memsw.usage_in_bytes.
+ - memory.memsw.limit_in_bytes.
+
+usage of mem+swap is limited by memsw.limit_in_bytes.
+
+Note: why 'mem+swap' rather than swap.
+The global LRU(kswapd) can swap out arbitrary pages. Swap-out means
+to move account from memory to swap...there is no change in usage of
+mem+swap.
+
+In other words, when we want to limit the usage of swap without affecting
+global LRU, mem+swap limit is better than just limiting swap from OS point
+of view.
+
+2.5 Reclaim
Each cgroup maintains a per cgroup LRU that consists of an active
and inactive list. When a cgroup goes over its limit, we first try
The memory.stat file gives accounting information. Now, the number of
caches, RSS and Active pages/Inactive pages are shown.
-The memory.force_empty gives an interface to drop *all* charges by force.
-
-# echo 1 > memory.force_empty
-
-will drop all charges in cgroup. Currently, this is maintained for test.
-
4. Testing
Balbir posted lmbench, AIM9, LTP and vmmstress results [10] and [11].
A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a
cgroup might have some charge associated with it, even though all
-tasks have migrated away from it. Such charges are automatically dropped at
-rmdir() if there are no tasks.
+tasks have migrated away from it.
+Such charges are freed(at default) or moved to its parent. When moved,
+both of RSS and CACHES are moved to parent.
+If both of them are busy, rmdir() returns -EBUSY. See 5.1 Also.
+
+Charges recorded in swap information is not updated at removal of cgroup.
+Recorded information is discarded and a cgroup which uses swap (swapcache)
+will be charged as a new owner of it.
+
+
+5. Misc. interfaces.
+
+5.1 force_empty
+ memory.force_empty interface is provided to make cgroup's memory usage empty.
+ You can use this interface only when the cgroup has no tasks.
+ When writing anything to this
+
+ # echo 0 > memory.force_empty
+
+ Almost all pages tracked by this memcg will be unmapped and freed. Some of
+ pages cannot be freed because it's locked or in-use. Such pages are moved
+ to parent and this cgroup will be empty. But this may return -EBUSY in
+ some too busy case.
+
+ Typical use case of this interface is that calling this before rmdir().
+ Because rmdir() moves all pages to parent, some out-of-use page caches can be
+ moved to the parent. If you want to avoid that, force_empty will be useful.
+
+5.2 stat file
+ memory.stat file includes following statistics (now)
+ cache - # of pages from page-cache and shmem.
+ rss - # of pages from anonymous memory.
+ pgpgin - # of event of charging
+ pgpgout - # of event of uncharging
+ active_anon - # of pages on active lru of anon, shmem.
+ inactive_anon - # of pages on active lru of anon, shmem
+ active_file - # of pages on active lru of file-cache
+ inactive_file - # of pages on inactive lru of file cache
+ unevictable - # of pages cannot be reclaimed.(mlocked etc)
+
+ Below is depend on CONFIG_DEBUG_VM.
+ inactive_ratio - VM inernal parameter. (see mm/page_alloc.c)
+ recent_rotated_anon - VM internal parameter. (see mm/vmscan.c)
+ recent_rotated_file - VM internal parameter. (see mm/vmscan.c)
+ recent_scanned_anon - VM internal parameter. (see mm/vmscan.c)
+ recent_scanned_file - VM internal parameter. (see mm/vmscan.c)
+
+ Memo:
+ recent_rotated means recent frequency of lru rotation.
+ recent_scanned means recent # of scans to lru.
+ showing for better debug please see the code for meanings.
+
+
+5.3 swappiness
+ Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only.
+
+ Following cgroup's swapiness can't be changed.
+ - root cgroup (uses /proc/sys/vm/swappiness).
+ - a cgroup which uses hierarchy and it has child cgroup.
+ - a cgroup which uses hierarchy and not the root of hierarchy.
+
+
+6. Hierarchy support
+
+The memory controller supports a deep hierarchy and hierarchical accounting.
+The hierarchy is created by creating the appropriate cgroups in the
+cgroup filesystem. Consider for example, the following cgroup filesystem
+hierarchy
+
+ root
+ / | \
+ / | \
+ a b c
+ | \
+ | \
+ d e
+
+In the diagram above, with hierarchical accounting enabled, all memory
+usage of e, is accounted to its ancestors up until the root (i.e, c and root),
+that has memory.use_hierarchy enabled. If one of the ancestors goes over its
+limit, the reclaim algorithm reclaims from the tasks in the ancestor and the
+children of the ancestor.
+
+6.1 Enabling hierarchical accounting and reclaim
+
+The memory controller by default disables the hierarchy feature. Support
+can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup
+
+# echo 1 > memory.use_hierarchy
+
+The feature can be disabled by
+
+# echo 0 > memory.use_hierarchy
+
+NOTE1: Enabling/disabling will fail if the cgroup already has other
+cgroups created below it.
+
+NOTE2: This feature can be enabled/disabled per subtree.
-5. TODO
+7. TODO
1. Add support for accounting huge pages (as a separate controller)
2. Make per-cgroup scanner reclaim not-shared pages first
3.6 Constraints
3.7 Example
-4 DRIVER DEVELOPER NOTES
+4 DMAENGINE DRIVER DEVELOPER NOTES
4.1 Conformance points
-4.2 "My application needs finer control of hardware channels"
+4.2 "My application needs exclusive control of hardware channels"
5 SOURCE
implementation examples.
4 DRIVER DEVELOPMENT NOTES
+
4.1 Conformance points:
There are a few conformance points required in dmaengine drivers to
accommodate assumptions made by applications using the async_tx API:
3/ Use async_tx_run_dependencies() in the descriptor clean up path to
handle submission of dependent operations
-4.2 "My application needs finer control of hardware channels"
-This requirement seems to arise from cases where a DMA engine driver is
-trying to support device-to-memory DMA. The dmaengine and async_tx
-implementations were designed for offloading memory-to-memory
-operations; however, there are some capabilities of the dmaengine layer
-that can be used for platform-specific channel management.
-Platform-specific constraints can be handled by registering the
-application as a 'dma_client' and implementing a 'dma_event_callback' to
-apply a filter to the available channels in the system. Before showing
-how to implement a custom dma_event callback some background of
-dmaengine's client support is required.
-
-The following routines in dmaengine support multiple clients requesting
-use of a channel:
-- dma_async_client_register(struct dma_client *client)
-- dma_async_client_chan_request(struct dma_client *client)
-
-dma_async_client_register takes a pointer to an initialized dma_client
-structure. It expects that the 'event_callback' and 'cap_mask' fields
-are already initialized.
-
-dma_async_client_chan_request triggers dmaengine to notify the client of
-all channels that satisfy the capability mask. It is up to the client's
-event_callback routine to track how many channels the client needs and
-how many it is currently using. The dma_event_callback routine returns a
-dma_state_client code to let dmaengine know the status of the
-allocation.
-
-Below is the example of how to extend this functionality for
-platform-specific filtering of the available channels beyond the
-standard capability mask:
-
-static enum dma_state_client
-my_dma_client_callback(struct dma_client *client,
- struct dma_chan *chan, enum dma_state state)
-{
- struct dma_device *dma_dev;
- struct my_platform_specific_dma *plat_dma_dev;
-
- dma_dev = chan->device;
- plat_dma_dev = container_of(dma_dev,
- struct my_platform_specific_dma,
- dma_dev);
-
- if (!plat_dma_dev->platform_specific_capability)
- return DMA_DUP;
-
- . . .
-}
+4.2 "My application needs exclusive control of hardware channels"
+Primarily this requirement arises from cases where a DMA engine driver
+is being used to support device-to-memory operations. A channel that is
+performing these operations cannot, for many platform specific reasons,
+be shared. For these cases the dma_request_channel() interface is
+provided.
+
+The interface is:
+struct dma_chan *dma_request_channel(dma_cap_mask_t mask,
+ dma_filter_fn filter_fn,
+ void *filter_param);
+
+Where dma_filter_fn is defined as:
+typedef bool (*dma_filter_fn)(struct dma_chan *chan, void *filter_param);
+
+When the optional 'filter_fn' parameter is set to NULL
+dma_request_channel simply returns the first channel that satisfies the
+capability mask. Otherwise, when the mask parameter is insufficient for
+specifying the necessary channel, the filter_fn routine can be used to
+disposition the available channels in the system. The filter_fn routine
+is called once for each free channel in the system. Upon seeing a
+suitable channel filter_fn returns DMA_ACK which flags that channel to
+be the return value from dma_request_channel. A channel allocated via
+this interface is exclusive to the caller, until dma_release_channel()
+is called.
+
+The DMA_PRIVATE capability flag is used to tag dma devices that should
+not be used by the general-purpose allocator. It can be set at
+initialization time if it is known that a channel will always be
+private. Alternatively, it is set when dma_request_channel() finds an
+unused "public" channel.
+
+A couple caveats to note when implementing a driver and consumer:
+1/ Once a channel has been privately allocated it will no longer be
+ considered by the general-purpose allocator even after a call to
+ dma_release_channel().
+2/ Since capabilities are specified at the device level a dma_device
+ with multiple channels will either have all channels public, or all
+ channels private.
5 SOURCE
-include/linux/dmaengine.h: core header file for DMA drivers and clients
+
+include/linux/dmaengine.h: core header file for DMA drivers and api users
drivers/dma/dmaengine.c: offload engine channel management routines
drivers/dma/: location for offload engine drivers
include/linux/async_tx.h: core header file for the async_tx api
justification is solid.
When making an incompatible API change, one should, whenever possible,
-ensure that code which has not been updated is caught by the compiler.
+ensure that code which has not been updated is caught by the compiler.
This will help you to be sure that you have found all in-tree uses of that
interface. It will also alert developers of out-of-tree code that there is
a change that they need to respond to. Supporting out-of-tree code is not
something that kernel developers need to be worried about, but we also do
-not have to make life harder for out-of-tree developers than it it needs to
-be.
+not have to make life harder for out-of-tree developers than it needs to
+be.
--- /dev/null
+See Documentation/crypto/async-tx-api.txt
void (*put_super) (struct super_block *);
void (*write_super) (struct super_block *);
int (*sync_fs)(struct super_block *sb, int wait);
- void (*write_super_lockfs) (struct super_block *);
- void (*unlockfs) (struct super_block *);
+ int (*freeze_fs) (struct super_block *);
+ int (*unfreeze_fs) (struct super_block *);
int (*statfs) (struct dentry *, struct kstatfs *);
int (*remount_fs) (struct super_block *, int *, char *);
void (*clear_inode) (struct inode *);
put_super: yes yes no
write_super: no yes read
sync_fs: no no read
-write_super_lockfs: ?
-unlockfs: ?
+freeze_fs: ?
+unfreeze_fs: ?
statfs: no no no
remount_fs: yes yes maybe (see below)
clear_inode: no
--- /dev/null
+
+ BTRFS
+ =====
+
+Btrfs is a new copy on write filesystem for Linux aimed at
+implementing advanced features while focusing on fault tolerance,
+repair and easy administration. Initially developed by Oracle, Btrfs
+is licensed under the GPL and open for contribution from anyone.
+
+Linux has a wealth of filesystems to choose from, but we are facing a
+number of challenges with scaling to the large storage subsystems that
+are becoming common in today's data centers. Filesystems need to scale
+in their ability to address and manage large storage, and also in
+their ability to detect, repair and tolerate errors in the data stored
+on disk. Btrfs is under heavy development, and is not suitable for
+any uses other than benchmarking and review. The Btrfs disk format is
+not yet finalized.
+
+The main Btrfs features include:
+
+ * Extent based file storage (2^64 max file size)
+ * Space efficient packing of small files
+ * Space efficient indexed directories
+ * Dynamic inode allocation
+ * Writable snapshots
+ * Subvolumes (separate internal filesystem roots)
+ * Object level mirroring and striping
+ * Checksums on data and metadata (multiple algorithms available)
+ * Compression
+ * Integrated multiple device support, with several raid algorithms
+ * Online filesystem check (not yet implemented)
+ * Very fast offline filesystem check
+ * Efficient incremental backup and FS mirroring (not yet implemented)
+ * Online filesystem defragmentation
+
+
+
+ MAILING LIST
+ ============
+
+There is a Btrfs mailing list hosted on vger.kernel.org. You can
+find details on how to subscribe here:
+
+http://vger.kernel.org/vger-lists.html#linux-btrfs
+
+Mailing list archives are available from gmane:
+
+http://dir.gmane.org/gmane.comp.file-systems.btrfs
+
+
+
+ IRC
+ ===
+
+Discussion of Btrfs also occurs on the #btrfs channel of the Freenode
+IRC network.
+
+
+
+ UTILITIES
+ =========
+
+Userspace tools for creating and manipulating Btrfs file systems are
+available from the git repository at the following location:
+
+ http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git
+ git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git
+
+These include the following tools:
+
+mkfs.btrfs: create a filesystem
+
+btrfsctl: control program to create snapshots and subvolumes:
+
+ mount /dev/sda2 /mnt
+ btrfsctl -s new_subvol_name /mnt
+ btrfsctl -s snapshot_of_default /mnt/default
+ btrfsctl -s snapshot_of_new_subvol /mnt/new_subvol_name
+ btrfsctl -s snapshot_of_a_snapshot /mnt/snapshot_of_new_subvol
+ ls /mnt
+ default snapshot_of_a_snapshot snapshot_of_new_subvol
+ new_subvol_name snapshot_of_default
+
+ Snapshots and subvolumes cannot be deleted right now, but you can
+ rm -rf all the files and directories inside them.
+
+btrfsck: do a limited check of the FS extent trees.
+
+btrfs-debug-tree: print all of the FS metadata in text form. Example:
+
+ btrfs-debug-tree /dev/sda2 >& big_output_file
# mount -t ext4 /dev/hda1 /wherever
- - When comparing performance with other filesystems, remember that
- ext3/4 by default offers higher data integrity guarantees than most.
- So when comparing with a metadata-only journalling filesystem, such
- as ext3, use `mount -o data=writeback'. And you might as well use
- `mount -o nobh' too along with it. Making the journal larger than
- the mke2fs default often helps performance with metadata-intensive
- workloads.
+ - When comparing performance with other filesystems, it's always
+ important to try multiple workloads; very often a subtle change in a
+ workload parameter can completely change the ranking of which
+ filesystems do well compared to others. When comparing versus ext3,
+ note that ext4 enables write barriers by default, while ext3 does
+ not enable write barriers by default. So it is useful to use
+ explicitly specify whether barriers are enabled or not when via the
+ '-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems
+ for a fair comparison. When tuning ext3 for best benchmark numbers,
+ it is often worthwhile to try changing the data journaling mode; '-o
+ data=writeback,nobh' can be faster for some workloads. (Note
+ however that running mounted with data=writeback can potentially
+ leave stale data exposed in recently written files in case of an
+ unclean shutdown, which could be a security exposure in some
+ situations.) Configuring the filesystem with a large journal can
+ also be helpful for metadata-intensive workloads.
2. Features
===========
* ability to use filesystems > 16TB (e2fsprogs support not available yet)
* extent format reduces metadata overhead (RAM, IO for access, transactions)
* extent format more robust in face of on-disk corruption due to magics,
-* internal redunancy in tree
+* internal redundancy in tree
* improved file allocation (multi-block alloc)
* fix 32000 subdirectory limit
* nsec timestamps for mtime, atime, ctime, create time
When mounting an ext4 filesystem, the following option are accepted:
(*) == default
-extents (*) ext4 will use extents to address file data. The
- file system will no longer be mountable by ext3.
-
-noextents ext4 will not use extents for newly created files
+ro Mount filesystem read only. Note that ext4 will
+ replay the journal (and thus write to the
+ partition) even when mounted "read only". The
+ mount options "ro,noload" can be used to prevent
+ writes to the filesystem.
journal_checksum Enable checksumming of the journal transactions.
This will allow the recovery code in e2fsck and the
journal=update Update the ext4 file system's journal to the current
format.
-journal=inum When a journal already exists, this option is ignored.
- Otherwise, it specifies the number of the inode which
- will represent the ext4 file system's journal file.
-
journal_dev=devnum When the external journal device's major/minor numbers
have changed, this option allows the user to specify
the new journal location. The journal device is
identified through its new major/minor numbers encoded
in devnum.
-noload Don't load the journal on mounting.
+noload Don't load the journal on mounting. Note that
+ if the filesystem was not unmounted cleanly,
+ skipping the journal replay will lead to the
+ filesystem containing inconsistencies that can
+ lead to any number of problems.
data=journal All data are committed into the journal prior to being
written into the main file system.
debug Extra debugging information is sent to syslog.
-errors=remount-ro(*) Remount the filesystem read-only on an error.
+errors=remount-ro Remount the filesystem read-only on an error.
errors=continue Keep going on a filesystem error.
errors=panic Panic and halt the machine if an error occurs.
+ (These mount options override the errors behavior
+ specified in the superblock, which can be configured
+ using tune2fs)
data_err=ignore(*) Just print an error message if an error occurs
in a file data buffer in ordered mode.
nodelalloc Disable delayed allocation. Blocks are allocation
when data is copied from user to page cache.
+max_batch_time=usec Maximum amount of time ext4 should wait for
+ additional filesystem operations to be batch
+ together with a synchronous write operation.
+ Since a synchronous write operation is going to
+ force a commit and then a wait for the I/O
+ complete, it doesn't cost much, and can be a
+ huge throughput win, we wait for a small amount
+ of time to see if any other transactions can
+ piggyback on the synchronous write. The
+ algorithm used is designed to automatically tune
+ for the speed of the disk, by measuring the
+ amount of time (on average) that it takes to
+ finish committing a transaction. Call this time
+ the "commit time". If the time that the
+ transactoin has been running is less than the
+ commit time, ext4 will try sleeping for the
+ commit time to see if other operati
projects / linux-2.6.git / commitdiff