1. 26 Oct, 2016 1 commit
    • Charlie Jacobsen's avatar
      Major overhaul of build process. · 8198c2fb
      Charlie Jacobsen authored
      Full kernel build no longer required. Yay! This should
      cut down on dev time a lot.
      
      I moved all of the LCD source into $(kernel-src)/lcd-domains/,
      so it's all in one spot. There is now a top-level makefile in
      there that triggers building liblcd, the microkernel, and the
      examples. This is built as an *external* build now, even
      though the directory is in the kernel source. The build now takes
      under a minute to do everything LCD related.
      
      This should also make verification easier in the future (e.g.
      building with clang) if we aren't ensnared in the kernel
      source.
      
      Of course, to use the microkernel and examples, you have to
      build the patched kernel and install it. But now when you
      make a few lines of changes in e.g. an example, you don't have
      to trigger a top-level kernel build to rebuild it. Running
      the full kernel build takes on average about 3 - 4 minutes
      (some files are generated everytime, linking is done, and so
      on), and can take upwards of 30 minutes for a full build if you
      re-config'd.
      
      Which brings me to my other change: no more config for LCDs
      in menuconfig. If we create menu entries for every example
      and so on, we end up changing the config too often, and this
      triggers full kernel rebuilds == waste of time. We can use
      macros by setting them via compiler flags (e.g., -DSOME_FLAG).
      Furthermore, it wasn't making sense to me to do conditional
      compilation for LCD support (we always want to compile for that).
      Yes, changes aren't clearly delineated with macros, but you can
      see changes made by just doing 'git diff v3.10.14 some-file-or-dir'.
      
      The wiki has been fully updated with instructions for building,
      and other relevant parts (updated paths to files).
      
      I also took the opportunity to clean up some old stuff lying around
      that is dead (like lcdguest). I incorporated all of the documentation
      in Documentation/lcd-domains into the wiki so it's all in one
      spot now (including some helpful debug tips).
      8198c2fb
  2. 25 Oct, 2016 1 commit
    • Charlie Jacobsen's avatar
      Linux slab allocator and page allocator inside LCDs. · 35ed57d7
      Charlie Jacobsen authored
      Single threaded, no locks, no fancy NUMA/percpu.
      
      Passing some simple examples. Added a memory management
      example module, in test-mods/mm, that exercises a lot
      of this new code.
      
      1:
      
      I moved in and adapted our existing guest virtual
      paging code from kliblcd.c. I'm using statically
      allocated bitmaps and arrays for tracking allocations
      in the guest virtual and physical address spaces.
      Using identity mapping for ease. (I decided not to
      use Linux's page allocator since it's too intertwined
      with the boot process - percpu variables, freeing
      init mem, boot allocator, all kinds of complexity ...)
      It might not be too hard to reimplement the buddy
      allocator algorithms, since I had to include a
      statically allocated array of struct pages anyway.
      
      I've set aside about 16 MBs for dynamic page allocations,
      but this can be changed using macros. You can allocate
      1, 2, 4, 8, etc. pages at a time. (The slab allocator
      requires this.)
      
      2:
      
      I finally broke down and set up boot info pages - 4
      boot pages right now, can be adjusted with a macro.
      Whoever boots the lcd needs to pack in information about the
      lcd's address space, initial cspace, and so on. 4 pages
      is enough to pack in information for larger modules
      like the mm example.
      
      3:
      
      I moved liblcd to a separate directory, and hacked the
      build system so that we can build liblcd as a static
      library and link it with example modules.
      
      liblcd/ contains lcd/, which has code for interacting
      with the microkernel and my simple page allocator.
      
      The Linux slab allocator is inside mm/, and some
      needed dependencies are in lib/. I made very few
      changes to the source code itself, but used some
      preprocessor/compiler hacks to make everything work. See
      Documentation/lcd-domains/liblcd.txt. I elided all of
      the locking and made it single core, single NUMA node.
      It's possible we'll see some bigs in the future, in code I haven't
      excerised yet (will probably manifest themselves as
      page faults).
      
      Ideally, we should have a separate tree for liblcd
      and building modules. That way we can avoid some of these
      hacks (maybe not all).
      
      4:
      
      Updated a lot of the documentation in
      Documentation/lcd-domains/.
      35ed57d7
  3. 22 Oct, 2016 1 commit
  4. 20 Oct, 2016 1 commit
  5. 16 Oct, 2016 1 commit
  6. 07 Oct, 2016 1 commit
  7. 02 Oct, 2016 1 commit
  8. 25 Sep, 2016 1 commit
  9. 18 Sep, 2016 1 commit
  10. 11 Sep, 2016 1 commit
  11. 04 Sep, 2016 1 commit
  12. 28 Aug, 2016 1 commit
  13. 21 Aug, 2016 1 commit
  14. 14 Aug, 2016 1 commit
  15. 08 Aug, 2016 1 commit
  16. 07 Aug, 2016 1 commit
  17. 02 Aug, 2016 1 commit
    • Steven Rostedt's avatar
      Makefile: Mute warning for __builtin_return_address(>0) for tracing only · 377ccbb4
      Steven Rostedt authored
      With the latest gcc compilers, they give a warning if
      __builtin_return_address() parameter is greater than 0. That is because if
      it is used by a function called by a top level function (or in the case of
      the kernel, by assembly), it can try to access stack frames outside the
      stack and crash the system.
      
      The tracing system uses __builtin_return_address() of up to 2! But it is
      well aware of the dangers that it may have, and has even added precautions
      to protect against it (see the thunk code in arch/x86/entry/thunk*.S)
      
      Linus originally added KBUILD_CFLAGS that would suppress the warning for the
      entire kernel, as simply adding KBUILD_CFLAGS to the tracing directory
      wouldn't work. The tracing directory plays a bit with the CFLAGS and
      requires a little more logic.
      
      This adds that special logic to only suppress the warning for the tracing
      directory. If it is used anywhere else outside of tracing, the warning will
      still be triggered.
      
      Link: http://lkml.kernel.org/r/20160728223043.51996267@grimm.local.homeTested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      377ccbb4
  18. 28 Jul, 2016 1 commit
  19. 27 Jul, 2016 2 commits
    • Linus Torvalds's avatar
      Disable "frame-address" warning · 124a3d88
      Linus Torvalds authored
      Newer versions of gcc warn about the use of __builtin_return_address()
      with a non-zero argument when "-Wall" is specified:
      
        kernel/trace/trace_irqsoff.c: In function ‘stop_critical_timings’:
        kernel/trace/trace_irqsoff.c:433:86: warning: calling ‘__builtin_return_address’ with a nonzero argument is unsafe [-Wframe-address]
           stop_critical_timing(CALLER_ADDR0, CALLER_ADDR1);
        [ .. repeats a few times for other similar cases .. ]
      
      It is true that a non-zero argument is somewhat dangerous, and we do not
      actually have very many uses of that in the kernel - but the ftrace code
      does use it, and as Stephen Rostedt says:
      
       "We are well aware of the danger of using __builtin_return_address() of
        > 0.  In fact that's part of the reason for having the "thunk" code in
        x86 (See arch/x86/entry/thunk_{64,32}.S).  [..] it adds extra frames
        when tracking irqs off sections, to prevent __builtin_return_address()
        from accessing bad areas.  In fact the thunk_32.S states: 'Trampoline to
        trace irqs off.  (otherwise CALLER_ADDR1 might crash)'."
      
      For now, __builtin_return_address() with a non-zero argument is the best
      we can do, and the warning is not helpful and can end up making people
      miss other warnings for real problems.
      
      So disable the frame-address warning on compilers that need it.
      Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      124a3d88
    • Linus Torvalds's avatar
      Disable "maybe-uninitialized" warning globally · 6e8d666e
      Linus Torvalds authored
      Several build configurations had already disabled this warning because
      it generates a lot of false positives.  But some had not, and it was
      still enabled for "allmodconfig" builds, for example.
      
      Looking at the warnings produced, every single one I looked at was a
      false positive, and the warnings are frequent enough (and big enough)
      that they can easily hide real problems that you don't notice in the
      noise generated by -Wmaybe-uninitialized.
      
      The warning is good in theory, but this is a classic case of a warning
      that causes more problems than the warning can solve.
      
      If gcc gets better at avoiding false positives, we may be able to
      re-enable this warning.  But as is, we're better off without it, and I
      want to be able to see the *real* warnings.
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      6e8d666e
  20. 26 Jul, 2016 2 commits
    • Kees Cook's avatar
      kbuild: abort build on bad stack protector flag · c965b105
      Kees Cook authored
      Before, the stack protector flag was sanity checked before .config had
      been reprocessed.  This meant the build couldn't be aborted early, and
      only a warning could be emitted followed later by the compiler blowing
      up with an unknown flag.  This has caused a lot of confusion over time,
      so this splits the flag selection from sanity checking and performs the
      sanity checking after the make has been restarted from a reprocessed
      .config, so builds can be aborted as early as possible now.
      
      Additionally moves the x86-specific sanity check to the same location,
      since it suffered from the same warn-then-wait-for-compiler-failure
      problem.
      
      Link: http://lkml.kernel.org/r/20160712223043.GA11664@www.outflux.netSigned-off-by: default avatarKees Cook <keescook@chromium.org>
      Cc: Michal Marek <mmarek@suse.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c965b105
    • Kees Cook's avatar
      kbuild: Abort build on bad stack protector flag · 228d96c6
      Kees Cook authored
      Before, the stack protector flag was sanity checked before .config had
      been reprocessed. This meant the build couldn't be aborted early, and
      only a warning could be emitted followed later by the compiler blowing
      up with an unknown flag. This has caused a lot of confusion over time,
      so this splits the flag selection from sanity checking and performs the
      sanity checking after the make has been restarted from a reprocessed
      .config, so builds can be aborted as early as possible now.
      
      Additionally moves the x86-specific sanity check to the same location,
      since it suffered from the same warn-then-wait-for-compiler-failure
      problem.
      Signed-off-by: default avatarKees Cook <keescook@chromium.org>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      228d96c6
  21. 24 Jul, 2016 1 commit
  22. 22 Jul, 2016 1 commit
    • Arnd Bergmann's avatar
      kbuild: make samples depend on headers_install · ddea05fa
      Arnd Bergmann authored
      Olof's build test setup keeps failing to compile arm64 kernels
      because of a toolchain that uses outdated kernel headers:
      
      /work/build/batch/samples/seccomp/bpf-fancy.c:13:27: fatal error: linux/seccomp.h: No such file or directory
      
      This is of course something he could change, but it also indicates
      that others may run into the same problem. Running 'make headers_install'
      avoids the issue by ensuring that the kernel headers are put into
      the $(objdir)/usr/include path before we build the samples.
      
      The same problem happened for the Documentation build in the
      past and was fixed up with commit 8e2faea8 ("Make Documenation
      depend on headers_install"). This adds an identical Makefile dependency
      for the samples/ subdirectory.
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      ddea05fa
  23. 18 Jul, 2016 2 commits
    • Arnd Bergmann's avatar
      Kbuild: always prefix objtree in LINUXINCLUDE · 3308b285
      Arnd Bergmann authored
      When $(LINUXINCLUDE) is added to the cflags of a target that
      normall doesn't have it (e.g. HOSTCFLAGS), each entry in the
      list is expanded so that we search both $(objtree) and $(srctree),
      which is a bit silly, as we already know which of the two we
      want for each entry in LINUXINCLUDE.
      
      Also, a follow-up patch changes the behavior so we only look in
      $(srctree) for manually added include path, and that breaks finding
      the generated headers.
      
      This adds an explicit $(objtree) for each tree that we want to
      look for generated files.
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      3308b285
    • Arnd Bergmann's avatar
      Kbuild: avoid duplicate include path · dc33db7c
      Arnd Bergmann authored
      arch/$(hdr-arch)/include/generated/uapi is included twice in the
      header search path, which is unnecessary, so this changes the
      top-level Makefile to drop the second instance by filtering out
      everything from USERINCLUDE that was already part of LINUXINCLUDE.
      
      This should have very little effect other than making the 'make V=1'
      output slightly smaller and making the build time faster by a miniscule
      amount, but it seems to be cleaner.
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      dc33db7c
  24. 10 Jul, 2016 1 commit
  25. 04 Jul, 2016 1 commit
  26. 01 Jul, 2016 1 commit
  27. 26 Jun, 2016 1 commit
  28. 23 Jun, 2016 1 commit
  29. 21 Jun, 2016 1 commit
  30. 19 Jun, 2016 1 commit
  31. 12 Jun, 2016 1 commit
  32. 07 Jun, 2016 3 commits
    • Michal Marek's avatar
      kbuild: Initialize exported variables · b36fad65
      Michal Marek authored
      The NOSTDINC_FLAGS variable is exported, so it needs to be cleared to
      avoid duplicating its content when running make from within make (e.g.
      in the packaging targets). This became an issue after commit
      9c8fa9bc ("kbuild: fix if_change and friends to consider argument
      order"), which no longer ignores the duplicate options. As Paulo Zanoni
      points out, the LDFLAGS_vmlinux variable has the same problem.
      Reported-by: default avatar"Zanoni, Paulo R" <paulo.r.zanoni@intel.com>
      Fixes: 9c8fa9bc ("kbuild: fix if_change and friends to consider argument order")
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      b36fad65
    • Emese Revfy's avatar
      Add sancov plugin · 543c37cb
      Emese Revfy authored
      The sancov gcc plugin inserts a __sanitizer_cov_trace_pc() call
      at the start of basic blocks.
      
      This plugin is a helper plugin for the kcov feature. It supports
      all gcc versions with plugin support (from gcc-4.5 on).
      It is based on the gcc commit "Add fuzzing coverage support" by Dmitry Vyukov
      (https://gcc.gnu.org/viewcvs/gcc?limit_changes=0&view=revision&revision=231296).
      Signed-off-by: default avatarEmese Revfy <re.emese@gmail.com>
      Acked-by: default avatarKees Cook <keescook@chromium.org>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      543c37cb
    • Emese Revfy's avatar
      GCC plugin infrastructure · 6b90bd4b
      Emese Revfy authored
      This patch allows to build the whole kernel with GCC plugins. It was ported from
      grsecurity/PaX. The infrastructure supports building out-of-tree modules and
      building in a separate directory. Cross-compilation is supported too.
      Currently the x86, arm, arm64 and uml architectures enable plugins.
      
      The directory of the gcc plugins is scripts/gcc-plugins. You can use a file or a directory
      there. The plugins compile with these options:
       * -fno-rtti: gcc is compiled with this option so the plugins must use it too
       * -fno-exceptions: this is inherited from gcc too
       * -fasynchronous-unwind-tables: this is inherited from gcc too
       * -ggdb: it is useful for debugging a plugin (better backtrace on internal
          errors)
       * -Wno-narrowing: to suppress warnings from gcc headers (ipa-utils.h)
       * -Wno-unused-variable: to suppress warnings from gcc headers (gcc_version
          variable, plugin-version.h)
      
      The infrastructure introduces a new Makefile target called gcc-plugins. It
      supports all gcc versions from 4.5 to 6.0. The scripts/gcc-plugin.sh script
      chooses the proper host compiler (gcc-4.7 can be built by either gcc or g++).
      This script also checks the availability of the included headers in
      scripts/gcc-plugins/gcc-common.h.
      
      The gcc-common.h header contains frequently included headers for GCC plugins
      and it has a compatibility layer for the supported gcc versions.
      
      The gcc-generate-*-pass.h headers automatically generate the registration
      structures for GIMPLE, SIMPLE_IPA, IPA and RTL passes.
      
      Note that 'make clean' keeps the *.so files (only the distclean or mrproper
      targets clean all) because they are needed for out-of-tree modules.
      
      Based on work created by the PaX Team.
      Signed-off-by: default avatarEmese Revfy <re.emese@gmail.com>
      Acked-by: default avatarKees Cook <keescook@chromium.org>
      Signed-off-by: default avatarMichal Marek <mmarek@suse.com>
      6b90bd4b
  33. 05 Jun, 2016 1 commit
  34. 30 May, 2016 1 commit
    • Jani Nikula's avatar
      Documentation/sphinx: add basic working Sphinx configuration and build · 22cba31b
      Jani Nikula authored
      Add basic configuration and makefile to build documentation from any
      .rst files under Documentation using Sphinx. For starters, there's just
      the placeholder index.rst.
      
      At the top level Makefile, hook Sphinx documentation targets alongside
      (but independent of) the DocBook toolchain, having both be run on the
      various 'make *docs' targets.
      
      All Sphinx processing is placed into Documentation/Makefile.sphinx. Both
      that and the Documentation/DocBook/Makefile are now expected to handle
      all the documentation targets, explicitly ignoring them if they're not
      relevant for that particular toolchain. The changes to the existing
      DocBook Makefile are kept minimal.
      
      There is graceful handling of missing Sphinx and rst2pdf (which is
      needed for pdf output) by checking for the tool and python module,
      respectively, with informative messages to the user.
      
      If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but
      otherwise gracefully fall back to the Sphinx default theme, with an
      informative message to the user, and slightly less pretty HTML output.
      
      Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available),
      epubdocs and xmldocs targets. The output documents are written into per
      output type subdirectories under Documentation/output.
      
      Finally, you can pass options to sphinx-build using the SPHINXBUILD make
      variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose
      output from Sphinx.
      
      This is based on the original work by Jonathan Corbet, but he probably
      wouldn't recognize this as his own anymore.
      Signed-off-by: default avatarJani Nikula <jani.nikula@intel.com>
      22cba31b
  35. 29 May, 2016 1 commit