1. 04 Jan, 2016 1 commit
  2. 25 Nov, 2015 1 commit
  3. 13 May, 2015 4 commits
  4. 14 Nov, 2014 1 commit
  5. 13 Nov, 2014 1 commit
  6. 02 May, 2014 1 commit
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Use rcu_dereference_sched() for trace event triggers · 561a4fe8
      Steven Rostedt (Red Hat) authored
      
      
      As trace event triggers are now part of the mainline kernel, I added
      my trace event trigger tests to my test suite I run on all my kernels.
      Now these tests get run under different config options, and one of
      those options is CONFIG_PROVE_RCU, which checks under lockdep that
      the rcu locking primitives are being used correctly. This triggered
      the following splat:
      
      ===============================
      [ INFO: suspicious RCU usage. ]
      3.15.0-rc2-test+ #11 Not tainted
      -------------------------------
      kernel/trace/trace_events_trigger.c:80 suspicious rcu_dereference_check() usage!
      
      other info that might help us debug this:
      
      rcu_scheduler_active = 1, debug_locks = 0
      4 locks held by swapper/1/0:
       #0:  ((&(&j_cdbs->work)->timer)){..-...}, at: [<ffffffff8104d2cc>] call_timer_fn+0x5/0x1be
       #1:  (&(&pool->lock)->rlock){-.-...}, at: [<ffffffff81059856>] __queue_work+0x140/0x283
       #2:  (&p->pi_lock){-.-.-.}, at: [<ffffffff8106e961>] try_to_wake_up+0x2e/0x1e8
       #3:  (&rq->lock){-.-.-.}, at: [<ffffffff8106ead3>] try_to_wake_up+0x1a0/0x1e8
      
      stack backtrace:
      CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.15.0-rc2-test+ #11
      Hardware name:                  /DG965MQ, BIOS MQ96510J.86A.0372.2006.0605.1717 06/05/2006
       0000000000000001 ffff88007e083b98 ffffffff819f53a5 0000000000000006
       ffff88007b0942c0 ffff88007e083bc8 ffffffff81081307 ffff88007ad96d20
       0000000000000000 ffff88007af2d840 ffff88007b2e701c ffff88007e083c18
      Call Trace:
       <IRQ>  [<ffffffff819f53a5>] dump_stack+0x4f/0x7c
       [<ffffffff81081307>] lockdep_rcu_suspicious+0x107/0x110
       [<ffffffff810ee51c>] event_triggers_call+0x99/0x108
       [<ffffffff810e8174>] ftrace_event_buffer_commit+0x42/0xa4
       [<ffffffff8106aadc>] ftrace_raw_event_sched_wakeup_template+0x71/0x7c
       [<ffffffff8106bcbf>] ttwu_do_wakeup+0x7f/0xff
       [<ffffffff8106bd9b>] ttwu_do_activate.constprop.126+0x5c/0x61
       [<ffffffff8106eadf>] try_to_wake_up+0x1ac/0x1e8
       [<ffffffff8106eb77>] wake_up_process+0x36/0x3b
       [<ffffffff810575cc>] wake_up_worker+0x24/0x26
       [<ffffffff810578bc>] insert_work+0x5c/0x65
       [<ffffffff81059982>] __queue_work+0x26c/0x283
       [<ffffffff81059999>] ? __queue_work+0x283/0x283
       [<ffffffff810599b7>] delayed_work_timer_fn+0x1e/0x20
       [<ffffffff8104d3a6>] call_timer_fn+0xdf/0x1be^M
       [<ffffffff8104d2cc>] ? call_timer_fn+0x5/0x1be
       [<ffffffff81059999>] ? __queue_work+0x283/0x283
       [<ffffffff8104d823>] run_timer_softirq+0x1a4/0x22f^M
       [<ffffffff8104696d>] __do_softirq+0x17b/0x31b^M
       [<ffffffff81046d03>] irq_exit+0x42/0x97
       [<ffffffff81a08db6>] smp_apic_timer_interrupt+0x37/0x44
       [<ffffffff81a07a2f>] apic_timer_interrupt+0x6f/0x80
       <EOI>  [<ffffffff8100a5d8>] ? default_idle+0x21/0x32
       [<ffffffff8100a5d6>] ? default_idle+0x1f/0x32
       [<ffffffff8100ac10>] arch_cpu_idle+0xf/0x11
       [<ffffffff8107b3a4>] cpu_startup_entry+0x1a3/0x213
       [<ffffffff8102a23c>] start_secondary+0x212/0x219
      
      The cause is that the triggers are protected by rcu_read_lock_sched() but
      the data is dereferenced with rcu_dereference() which expects it to
      be protected with rcu_read_lock(). The proper reference should be
      rcu_dereference_sched().
      
      Cc: Tom Zanussi <tom.zanussi@linux.intel.com>
      Cc: stable@vger.kernel.org # 3.14+
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      561a4fe8
  7. 08 Apr, 2014 1 commit
  8. 09 Jan, 2014 2 commits
  9. 02 Jan, 2014 2 commits
  10. 21 Dec, 2013 4 commits
    • Tom Zanussi's avatar
      tracing: Add and use generic set_trigger_filter() implementation · bac5fb97
      Tom Zanussi authored
      Add a generic event_command.set_trigger_filter() op implementation and
      have the current set of trigger commands use it - this essentially
      gives them all support for filters.
      
      Syntactically, filters are supported by adding 'if <filter>' just
      after the command, in which case only events matching the filter will
      invoke the trigger.  For example, to add a filter to an
      enable/disable_event command:
      
          echo 'enable_event:system:event if common_pid == 999' > \
                    .../othersys/otherevent/trigger
      
      The above command will only enable the system:event event if the
      common_pid field in the othersys:otherevent event is 999.
      
      As another example, to add a filter to a stacktrace command:
      
          echo 'stacktrace if common_pid == 999' > \
                         .../somesys/someevent/trigger
      
      The above command will only trigger a stacktrace if the common_pid
      field in the event is 999.
      
      The filter syntax is the same as that described in the 'Event
      filtering' section of Documentation/trace/events.txt.
      
      Because triggers can now use filters, the trigger-invoking logic needs
      to be moved in those cases - e.g. for ftrace_raw_event_calls, if a
      trigger has a filter associated with it, the trigger invocation now
      needs to happen after the { assign; } part of the call, in order for
      the trigger condition to be tested.
      
      There's still a SOFT_DISABLED-only check at the top of e.g. the
      ftrace_raw_events function, so when an event is soft disabled but not
      because of the presence of a trigger, the original SOFT_DISABLED
      behavior remains unchanged.
      
      There's also a bit of trickiness in that some triggers need to avoid
      being invoked while an event is currently in the process of being
      logged, since the trigger may itself log data into the trace buffer.
      Thus we make sure the current event is committed before invoking those
      triggers.  To do that, we split the trigger invocation in two - the
      first part (event_triggers_call()) checks the filter using the current
      trace record; if a command has the post_trigger flag set, it sets a
      bit for itself in the return value, otherwise it directly invoks the
      trigger.  Once all commands have been either invoked or set their
      return flag, event_triggers_call() returns.  The current record is
      then either committed or discarded; if any commands have deferred
      their triggers, those commands are finally invoked following the close
      of the current event by event_triggers_post_call().
      
      To simplify the above and make it more efficient, the TRIGGER_COND bit
      is introduced, which is set only if a soft-disabled trigger needs to
      use the log record for filter testing or needs to wait until the
      current log record is closed.
      
      The syscall event invocation code is also changed in analogous ways.
      
      Because event triggers need to be able to create and free filters,
      this also adds a couple external wrappers for the existing
      create_filter and free_filter functions, which are too generic to be
      made extern functions themselves.
      
      Link: http://lkml.kernel.org/r/7164930759d8719ef460357f143d995406e4eead.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      bac5fb97
    • Tom Zanussi's avatar
      tracing: Add 'enable_event' and 'disable_event' event trigger commands · 7862ad18
      Tom Zanussi authored
      Add 'enable_event' and 'disable_event' event_command commands.
      
      enable_event and disable_event event triggers are added by the user
      via these commands in a similar way and using practically the same
      syntax as the analagous 'enable_event' and 'disable_event' ftrace
      function commands, but instead of writing to the set_ftrace_filter
      file, the enable_event and disable_event triggers are written to the
      per-event 'trigger' files:
      
          echo 'enable_event:system:event' > .../othersys/otherevent/trigger
          echo 'disable_event:system:event' > .../othersys/otherevent/trigger
      
      The above commands will enable or disable the 'system:event' trace
      events whenever the othersys:otherevent events are hit.
      
      This also adds a 'count' version that limits the number of times the
      command will be invoked:
      
          echo 'enable_event:system:event:N' > .../othersys/otherevent/trigger
          echo 'disable_event:system:event:N' > .../othersys/otherevent/trigger
      
      Where N is the number of times the command will be invoked.
      
      The above commands will will enable or disable the 'system:event'
      trace events whenever the othersys:otherevent events are hit, but only
      N times.
      
      This also makes the find_event_file() helper function extern, since
      it's useful to use from other places, such as the event triggers code,
      so make it accessible.
      
      Link: http://lkml.kernel.org/r/f825f3048c3f6b026ee37ae5825f9fc373451828.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      7862ad18
    • Tom Zanussi's avatar
      tracing: Add 'stacktrace' event trigger command · f21ecbb3
      Tom Zanussi authored
      Add 'stacktrace' event_command.  stacktrace event triggers are added
      by the user via this command in a similar way and using practically
      the same syntax as the analogous 'stacktrace' ftrace function command,
      but instead of writing to the set_ftrace_filter file, the stacktrace
      event trigger is written to the per-event 'trigger' files:
      
          echo 'stacktrace' > .../tracing/events/somesys/someevent/trigger
      
      The above command will turn on stacktraces for someevent i.e. whenever
      someevent is hit, a stacktrace will be logged.
      
      This also adds a 'count' version that limits the number of times the
      command will be invoked:
      
          echo 'stacktrace:N' > .../tracing/events/somesys/someevent/trigger
      
      Where N is the number of times the command will be invoked.
      
      The above command will log N stacktraces for someevent i.e. whenever
      someevent is hit N times, a stacktrace will be logged.
      
      Link: http://lkml.kernel.org/r/0c30c008a0828c660aa0e1bbd3255cf179ed5c30.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      f21ecbb3
    • Tom Zanussi's avatar
      tracing: Add 'snapshot' event trigger command · 93e31ffb
      Tom Zanussi authored
      Add 'snapshot' event_command.  snapshot event triggers are added by
      the user via this command in a similar way and using practically the
      same syntax as the analogous 'snapshot' ftrace function command, but
      instead of writing to the set_ftrace_filter file, the snapshot event
      trigger is written to the per-event 'trigger' files:
      
          echo 'snapshot' > .../somesys/someevent/trigger
      
      The above command will turn on snapshots for someevent i.e. whenever
      someevent is hit, a snapshot will be done.
      
      This also adds a 'count' version that limits the number of times the
      command will be invoked:
      
          echo 'snapshot:N' > .../somesys/someevent/trigger
      
      Where N is the number of times the command will be invoked.
      
      The above command will snapshot N times for someevent i.e. whenever
      someevent is hit N times, a snapshot will be done.
      
      Also adds a new tracing_alloc_snapshot() function - the existing
      tracing_snapshot_alloc() function is a special version of
      tracing_snapshot() that also does the snapshot allocation - the
      snapshot triggers would like to be able to do just the allocation but
      not take a snapshot; the existing tracing_snapshot_alloc() in turn now
      also calls tracing_alloc_snapshot() underneath to do that allocation.
      
      Link: http://lkml.kernel.org/r/c9524dd07ce01f9dcbd59011290e0a8d5b47d7ad.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      [ fix up from kbuild test robot <fengguang.wu@intel.com report ]
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      93e31ffb
  11. 20 Dec, 2013 2 commits
    • Tom Zanussi's avatar
      tracing: Add 'traceon' and 'traceoff' event trigger commands · 2a2df321
      Tom Zanussi authored
      Add 'traceon' and 'traceoff' event_command commands.  traceon and
      traceoff event triggers are added by the user via these commands in a
      similar way and using practically the same syntax as the analagous
      'traceon' and 'traceoff' ftrace function commands, but instead of
      writing to the set_ftrace_filter file, the traceon and traceoff
      triggers are written to the per-event 'trigger' files:
      
          echo 'traceon' > .../tracing/events/somesys/someevent/trigger
          echo 'traceoff' > .../tracing/events/somesys/someevent/trigger
      
      The above command will turn tracing on or off whenever someevent is
      hit.
      
      This also adds a 'count' version that limits the number of times the
      command will be invoked:
      
          echo 'traceon:N' > .../tracing/events/somesys/someevent/trigger
          echo 'traceoff:N' > .../tracing/events/somesys/someevent/trigger
      
      Where N is the number of times the command will be invoked.
      
      The above commands will will turn tracing on or off whenever someevent
      is hit, but only N times.
      
      Some common register/unregister_trigger() implementations of the
      event_command reg()/unreg() callbacks are also provided, which add and
      remove trigger instances to the per-event list of triggers, and
      arm/disarm them as appropriate.  event_trigger_callback() is a
      general-purpose event_command func() implementation that orchestrates
      command parsing and registration for most normal commands.
      
      Most event commands will use these, but some will override and
      possibly reuse them.
      
      The event_trigger_init(), event_trigger_free(), and
      event_trigger_print() functions are meant to be common implementations
      of the event_trigger_ops init(), free(), and print() ops,
      respectively.
      
      Most trigger_ops implementations will use these, but some will
      override and possibly reuse them.
      
      Link: http://lkml.kernel.org/r/00a52816703b98d2072947478dd6e2d70cde5197.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      2a2df321
    • Tom Zanussi's avatar
      tracing: Add basic event trigger framework · 85f2b082
      Tom Zanussi authored
      Add a 'trigger' file for each trace event, enabling 'trace event
      triggers' to be set for trace events.
      
      'trace event triggers' are patterned after the existing 'ftrace
      function triggers' implementation except that triggers are written to
      per-event 'trigger' files instead of to a single file such as the
      'set_ftrace_filter' used for ftrace function triggers.
      
      The implementation is meant to be entirely separate from ftrace
      function triggers, in order to keep the respective implementations
      relatively simple and to allow them to diverge.
      
      The event trigger functionality is built on top of SOFT_DISABLE
      functionality.  It adds a TRIGGER_MODE bit to the ftrace_event_file
      flags which is checked when any trace event fires.  Triggers set for a
      particular event need to be checked regardless of whether that event
      is actually enabled or not - getting an event to fire even if it's not
      enabled is what's already implemented by SOFT_DISABLE mode, so trigger
      mode directly reuses that.  Event trigger essentially inherit the soft
      disable logic in __ftrace_event_enable_disable() while adding a bit of
      logic and trigger reference counting via tm_ref on top of that in a
      new trace_event_trigger_enable_disable() function.  Because the base
      __ftrace_event_enable_disable() code now needs to be invoked from
      outside trace_events.c, a wrapper is also added for those usages.
      
      The triggers for an event are actually invoked via a new function,
      event_triggers_call(), and code is also added to invoke them for
      ftrace_raw_event calls as well as syscall events.
      
      The main part of the patch creates a new trace_events_trigger.c file
      to contain the trace event triggers implementation.
      
      The standard open, read, and release file operations are implemented
      here.
      
      The open() implementation sets up for the various open modes of the
      'trigger' file.  It creates and attaches the trigger iterator and sets
      up the command parser.  If opened for reading set up the trigger
      seq_ops.
      
      The read() implementation parses the event trigger written to the
      'trigger' file, looks up the trigger command, and passes it along to
      that event_command's func() implementation for command-specific
      processing.
      
      The release() implementation does whatever cleanup is needed to
      release the 'trigger' file, like releasing the parser and trigger
      iterator, etc.
      
      A couple of functions for event command registration and
      unregistration are added, along with a list to add them to and a mutex
      to protect them, as well as an (initially empty) registration function
      to add the set of commands that will be added by future commits, and
      call to it from the trace event initialization code.
      
      also added are a couple trigger-specific data structures needed for
      these implementations such as a trigger iterator and a struct for
      trigger-specific data.
      
      A couple structs consisting mostly of function meant to be implemented
      in command-specific ways, event_command and event_trigger_ops, are
      used by the generic event trigger command implementations.  They're
      being put into trace.h alongside the other trace_event data structures
      and functions, in the expectation that they'll be needed in several
      trace_event-related files such as trace_events_trigger.c and
      trace_events.c.
      
      The event_command.func() function is meant to be called by the trigger
      parsing code in order to add a trigger instance to the corresponding
      event.  It essentially coordinates adding a live trigger instance to
      the event, and arming the triggering the event.
      
      Every event_command func() implementation essentially does the
      same thing for any command:
      
         - choose ops - use the value of param to choose either a number or
           count version of event_trigger_ops specific to the command
         - do the register or unregister of those ops
         - associate a filter, if specified, with the triggering event
      
      The reg() and unreg() ops allow command-specific implementations for
      event_trigger_op registration and unregistration, and the
      get_trigger_ops() op allows command-specific event_trigger_ops
      selection to be parameterized.  When a trigger instance is added, the
      reg() op essentially adds that trigger to the triggering event and
      arms it, while unreg() does the opposite.  The set_filter() function
      is used to associate a filter with the trigger - if the command
      doesn't specify a set_filter() implementation, the command will ignore
      filters.
      
      Each command has an associated trigger_type, which serves double duty,
      both as a unique identifier for the command as well as a value that
      can be used for setting a trigger mode bit during trigger invocation.
      
      The signature of func() adds a pointer to the event_command struct,
      used to invoke those functions, along with a command_data param that
      can be passed to the reg/unreg functions.  This allows func()
      implementations to use command-specific blobs and supports code
      re-use.
      
      The event_trigger_ops.func() command corrsponds to the trigger 'probe'
      function that gets called when the triggering event is actually
      invoked.  The other functions are used to list the trigger when
      needed, along with a couple mundane book-keeping functions.
      
      This also moves event_file_data() into trace.h so it can be used
      outside of trace_events.c.
      
      Link: http://lkml.kernel.org/r/316d95061accdee070aac8e5750afba0192fa5b9.1382622043.git.tom.zanussi@linux.intel.com
      
      Signed-off-by: default avatarTom Zanussi <tom.zanussi@linux.intel.com>
      Idea-by: default avatarSteve Rostedt <rostedt@goodmis.org>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      85f2b082