kabitools ========= This is a toolkit that is intended to assist in determining whether a given symbol affects or is affected by other symbols in a kABI sense. You must install boost to run this kit. What is a symbol? From the sparse project: An identifier with semantic meaning is a "symbol". There's a 1:n relationship: each symbol is always associated with one identifier, while each identifier can have one or more semantic meanings due to C scope rules. The progression is symbol -> token -> identifier. The token contains the information on where the symbol was declared. The kernel can be compiled so that a graph is made for each of the preprocessor ".i" files using the information supplied by sparse. The kabitools can be installed from the following website. Choose the rpm that matches your distro and architecture. http://people.redhat.com/tcamuso/kabitools/ The following will be installed by the rpm. /usr/sbin/kabi-data.sh /usr/sbin/kabi-dump /usr/sbin/kabi-graph /usr/sbin/kabi-lookup /usr/sbin/kabi-parser /usr/sbin/kabiscan /usr/sbin/kabiscan.help /usr/sbin/makei.sh /usr/share/doc/kabitools /usr/share/doc/kabitools/README /usr/share/kabitools-pegas-kernel-make.patch /usr/share/kabitools-rhel-kernel-make.patch Use of this toolkit consists primarily of two steps. 1. Create the kernel graph files 2. Use the lookup tool, kabi-lookup, or its menu-driven wrapper, kabiscan, to determine whether a given symbol could affect kABI. There are other tools in the kit for those who like to get under the hood, but the most practical application is comprised of the above two steps. --------------------------------------------------------------------------- Creating the Kernel Graph Files wiith the kabi-graph script --------------------------------------------------------------------------- This is the easiest way to create the kernel graph files. This script must be invoked from the top of the kernel tree. Time to complete a full kernel build is lengthened by about 10% by invoking this tool. kabi-graph -[jnDh] [target] Builds a graph from the preprocessor output of each file compiled. Must be invoked from the top of the kernel tree. The optional target argument can be any valid kernel make target including a subdirectory, e.g. "virt/" or a module build, e.g. "M=drivers/char/tpm" The kABI graph will be limited to the scope of the build of the optional target. Default action without the optarg: make clean # can be disabled with -n option make make modules Options -j - Optional number of processors to assign to the make task. There must be a space between -j and the number. Default is all processors in this system: 40 -n - Do not make clean first -R - Recreate the list of graph files: redhat/kabi/kabi-datafiles.list -D - Delete all the graph files and exit. -h - This help screen. NOTE: The graph file extension will be added to .git/info/exclude, but will be selectively removed with "kabi-graph -D" command, which removes all the graph files. There are other methods for those who like to tinker. They are detailed at the end of this README. --------------------------------------------------------------------------- Using kabiscan to Find Symbols Affecting or Affected by kABI --------------------------------------------------------------------------- The easiest way of doing this is with the kabiscan menu-driven wrapper for kabi-lookup. Before using this tool, the following criteria must be met. * Install boost. * Build the kernel graph using the kabi-graph script. * Build the kABI whitelist with "make rh-kabi". * This script must be invoked from the top of your kernel tree. Here is a view of the default menu after a fresh install of kabitools. Scan The Kernel for kABI Sensitive Symbols Environment Most Recent Tag : kernel-pegas-4.10.0-7.el7$ b Current git branch : pegas f File to receive output : /dev/stdout n Search name : m Search mask : t struct or union : struct Run Parameters 1 Stop after finding one : true l Only seek whitelist symbols : true w Whole words only : true v Verbose output : false s Search mode : kABI Sensitive Symbols Control r Run the search h help text q Quit Enter one of the above: Environment b - An enumerated list of git branches will be presented from which to select by number. f - Send the output to a file, instead of stdout. n - The name of the symbol being sought, e.g "device" or "enable_irq". m - An optional mask that can help speed searches, if you have an idea of where to look, e.g. "acpi" or "fs" t - This option does not appear when the Search Mode is set to "Exported Functions" or when the "Whole Word" search is disabled. By typing the associated key, the value toggles between "struct" and "union". This is used for more precise searches, when you know exactly what you're seeking. Run Parameters These parameters are toggled by pressing the associated key. 1 - Stop after finding the first instance when "Whole Word" mode is true. When "Whole Word" is false, the search stops after the first file containing strings that match the "Search Name." l - Only seek symbols that are in the kABI whitelists. If the symbol cannot be found in the white list, the search stops there. With this option true, the "Whole Word" option is automatically set to "true", since you should know exactly what you're seeking in the white list. w - Whole words only works similarly to the grep -w option. The is more precise with this option. v - Verbose mode will drill down through the whole hierarchy of the given symbol. This generates a lot of output, so don't use it unless you really need it. s - Search Mode steps through the four available search modes. Each time the key is pressed, the Search Mode advances to the next mode in the rotation. There are several search modes. Exported Functions Simple lookup of exported function names. Searching the white list requires an exact name. kABI Sensitive Symbols These are symbols like "struct device" that may or may not be kABI sensitive, but this mode will test to see if indeed it is. Searches can be inexact by setting the "Whole Word" option to false. Symbol Definition Seeks the actual definition of the symbol. Most useful when you just want a quick look at the members of a data structure. In verbose mode, the entire hierarchy of the structure is presented, right down to the last subordinate element. This is a lot of output, but when you need it, it's there. Count symbol instances Counts the number of times the symbol, or substring, appears in the graph. --------------------------------------------------------------------------- Using kabi-lookup to Find Symbols Affecting or Affected by kABI --------------------------------------------------------------------------- If you prefer the command line, you can use the kabi-lookup tool directly. kabi-lookup enables the user to determine whether a compound data type, i.e. struct or union, has any relation to exported symbols. Conversely, the tool can search for exported symbols to see their dependency tree. The tool traverses the graph files looking for the symbol provided at the command line. Searches can be limited to whitelists and partial or complete pathnames. If the given symbol is not a dependency of an exported symbol, a message is printed to stdout indicating that the symbol cannot be found. This does not mean that the symbol does not exist, but that the symbol does not appear anywhere in the hierarchy of any of the exported symbols in the search. It is also useful for dumping the symbols upon which an exported function depends. The default output for exported functions lists the name, the return value, if not void, and the argument list with types, if not void. You can even dump the members of a union or data structure. With the -v option, every symbol descendant from the symbol given at the command line is printed to stdout. This is a LOT of stuff, so it's better to redirect the output to a file for parsing. There are not many situations where this volume and depth of information is necessary, but it's there if you need it. You can also use the tool to count how many times a given symbol appears in the dependency trees of the exported symbols in the search. kabi-lookup [-vwl] -e|s|c|d symbol [-m mask] [-p path] Options: -e symbol Searches for an exported symbol. Strings that omit the -w switch will generate output from any and all symbols that contain the string. -s symbol Searches for the symbol of a compound data type that matches the string. -d symbol Search for the declaration of a compound data type and print its members to the screen -c symbol Counts the number of times the symbol occurs in the search. This is most useful when you know the exact name of the symbol and are using the -w switch. -v Verbose output, prints the hierarchy down to the lowest level. All descendants of nonscalar types are printed. This creates a LOT of data, so it's probably better to direct output to a text file for parsing. -w Whole words only. Default is substring match. NOTE: With this switch, if the symbol is a struct or union, the string must include the word "struct" or "union" before the symbol being searched, and the string must be encapsulated with single or double quotes. Examples: kabi-lookup -sw "struct pci_dev" kabi-lookup -sw 'union ipmi_smi_info_union' -l Limit the search to symbols in the white lists. What this means is that the symbol will only be considered found if it appears in the hierarchy of a white listed symbol. Run "make rh-kabi" to create the white lists. NOTE: Must be used with the -w switch, as described above, or the symbol will not be found. Examples: kabi-lookup -lsw "struct pci_dev" kabi-lookup -lsw 'union ipmi_smi_info_union' -m string The mask string limits the search to pathnames containing the mask string. For example ... kabi-lookup -m fs -sw 'struct device' ... will search every path having "fs" in its name, e.g. drivers/video/fbsysfs.i and fs/coredump.i If you wish to limit the search to a specific directory, a trailing / must be used, e.g. kabi-lookup -m fs/ -sw 'struct device' kabi-lookup -m drivers/acpi/ -sw 'struct acpi_device' The more unique the -m string, the more limited the search. -1 Hyphen-number-1, limits the search to the first instance. -p path Path to the top of the kernel tree, if executing in a different directory. ------------------- Some Usage Examples ------------------- $ kabi-lookup -s pci_dev Searches for all instances of "pci_dev", including such patterns as "struct pci_device_id". $ kabi-lookup -sw "struct pci_dev" Will search only for exactly "struct pci_dev". Whole word matches for compound data types must include the "struct" or "union" declaration. NOTE: Quotes are required around search patterns having spaces. $ kabi-lookup -m pci -s pci_bus Searches for all instances of the string "pci_bus" in any and all subdirectories and files that have "pci" in their name. $ kabi-lookup -e mlx4_cq Searches for any and all exported symbols that match "mlx4_cq" and their argument lists and return values. $ kabi-lookup -ew mlx4_cq_modify Will find the one and only unique mlx4_cq_modify exported symbol and its argument lists and return values. --------------------------------------------------------------------------- Other tools in the kabitools kit --------------------------------------------------------------------------- kabi-parser - Parses preprocessor .i files for exported symbols and the compound data types on which they depend. The results for each .i file are serialized into a data file. Exported symbols and their entire dependency trees are recorded. The data files have the same name stem and reside in the same directory as their .i sources. The data is organized as as a map using a symbol's crc and its place in the hierarchy of its dependent exported symbol. Compound data types will include their backpointers, which introduces a bit of redundancy, but maximizes the data. Backpointers are followed to only one level below to avoid infinite recursion. Nested duplicates are also detected and limited for the same reason. /usr/sbin/kabi-parser kabi-dump - Dumps the contents of a serialized data file to make it humanly readable. It is really only a debugging tool. /usr/sbin/kabi-dump makei.sh - Uses the kernel make to compile preprocessor .i files. To save time, only files that have EXPORT_SYMBOL in them are processed. /usr/sbin/makei.sh kabi-data.sh - A shell script wrapper for kabi-parser that invokes it for each .i file in the kernel tree. This shell script creates a list of files processed to be used by kabi-lookup tool. /usr/sbin/kabi-data.sh --------------------------------------------------------------------------- Other Methods For Creating the Kernel Graph Files --------------------------------------------------------------------------- 1. Patch the kernel Makefiles ----------------------------- Patches for the kernel Makefiles are also provided in this kit and are saved in /usr/share in the following files. /usr/share/kabitools-pegas-kernel-make.patch /usr/share/kabitools-rhel-kernel-make.patch The kabitools script actually applies the appropriate patch to build the kernel graph, and reverses the patch before exiting. Example: # Patch the kernel make patch -p1 < /usr/share/kabitools-rhel-kernel-make.patch When you invoke a kernel make with K=1, the graph of the kernel will be created during the compile of the kernel. Example: # Compile the kernel and create the kernel graph make -j16 K=1 From this point, kabi-lookup can be used as described above. 2. makei.sh and kabi-data.sh scripts ------------------------------------ This method of creating the kernel graph requires both these scripts. makei.sh - Use this script first to create the preprocessor intermediate ".i" files for the kernel or kernel subdirectory. Must be executed from the top of the kernel tree. Examples: # create .i files for the whole kernel makei.sh ./ # create .i files for all files under drivers/acpi makei.sh drivers/acpi kabi-data.sh - This script creates the graph files from the .i files It requires only one argument, the path to the top of the kernel tree. Other arguments can be seen by entering the command without arguments or with -h switch. The most useful of these is -S, which allows you to create a graph for just a kernel subdirectory. Examples: # Create the graph files for the whole kernel from the top # of the kernel tree. kabi-data.sh -d ./ # Create the graph files for the whole kernel from a # different PWD. kabi-data.sh -d /path/to/my/kernel/tree # Create the graph files for a subdirectory of the kernel. kabi-data.sh -d ./ -s drivers/acpi