First of all, I must admit that I fully agree with your "I've found this shockingly hard to do" statement. Google has designed Android mainly from a consumer perspective, and not for power users. The result is that, as soon as you want to do something outside of using the latest Facebook app or playing with Candy Crush, you very quickly find yourself back in realm of early-2000 Linux, when a developer-like knowledge was required to change what should be simple settings. I believe that the situation will fastly evolve as Android system gets more mature, but for now we have to do with what we have got...
As you said, there are two reasons why it is necessary to compile your own SELinux toolset:
- The system provided toolset is usually a version behind. While Android's SELinux relies on policy DB version 30, current Linux boxes usually handle only version up to 29.
- Even if it would be more recent it would not help, in fact building SELinux from upstream code (which is easilly done, at least on Fedora machines following upstream recommendations) effectively allows the system to handle policy DB version 30, however Android's SELinux has been heavilly modified (Google documentation highlights a few modifications) so trying to handle Android's SELinux fails due to syntax and parsing errors.
So, to keep on the Android's SELinux analysis quest, we will have to put our hands in the dirt... in the cleanest possible way:
- First we will setup a sane environment,
- Once this is done we will compile Android's SELinux libraries and first tools,
- On top of them we will build SELinux tools,
- We will finish by adding a few supplementary utilities.
Setup a proper environment
Environment properties
The cleanest recommended, an possibly maybe only reliably working way is to dedicate a environment to your Android work:
A virtual machine is perfectly fine (if not the best option). Prefer to use a VMware one since you will have to connect your phone through USB to the guest system. The free alternative Qemu doesn't seem to handle such task very well. I did not try with other virualization software.
It will need to be a 64 bits system, otherwise the code will simply not compile due to integers being of the wrong size.
It is strongly recommended, possibly mandatory, to use a Ubuntu system. Feel free to use Xubuntu instead if you prefer XFCE's lighter desktop environment, this does not change the system's core and available package and will have no impact on your Android related work (whatever I say about Ubuntu in this procedure also applies to Xubuntu). You may find in Android's SELinux source tree some ReadMe files recommending the use of Fedora instead, these files are inherited from upstream NSA's SELinux project and their content do not necessarily match Google's Android.
The exact version of Unbuntu to use depends on the version of Android you want to build. For Android 6.0, Ubuntu 14.04 (Trusty) is recommended. Check Google requirements page for more information.
You will need plenty of disk space (at least 50GB if you plan only SELinux-related investigation, at least 100GB if you plan for a complete build of Android). CPU and memory are less relevant, they only impact time for a full build and will have no real impact for SELinux related tasks.
Using Ubuntu has two main advantages:
By using the recommended system, you are working in a well-known and well-tested environment: system libraries, tools and packages are at the version and location expected by the project.
And more specifically in our current case: Ubuntu itself relies on AppArmor which is a SELinux alternative, it does not use SELinux. The good news is that you will therefore be able to install Android's SELinux tools and binaries system-wide without risking to alter system reliability.
Environment installation procedure
You can install Ubuntu the traditional way by starting from a full-fledged live-DVD, but a faster alternative is to use a netboot install (textmode install) and select the desktop environment you prefer at the end. Doing so will save you the initial update time by directly installing up-to-date packages version instead of first installing obsolete ones, then asking to apply 389 pending updates on the first boot.
The ISO for Ubuntu/Xubuntu 14.04 (same ISO) netboot installer is available here.
To skip VMware's troublesome "Easy Install" feature, it's a good habit to start by selecting the "I will install the operating system later" option.
Be sure to select Linux, then Ubuntu 64 bits as guest OS.
The VM will need the following ressources:
- Mandatory: disk space must be at the very least 40GB (the default 20 GB will not be enough, the source code alone takes more space than that), higher is recommended. A full build requires a 100 GB disk minimum, this is the value I usually take. Do not forget that this setting is just a maximum limit: the actual size taken by the VM grows dynamically with guest's requests.
- Facultative: Increase RAM from 1024 to at least 2048 or higher (depends on your host capacity, I use 4096),
- Facultative: Increase the number of processor cores from 1 to 2 or higher (depends on your host capacity, I use 3).
- The CD-Rom must point to the installation ISO file.
- You may want to switch USB from the default 1.1 to 2.0 as the former may give warnings when you connect your device. Depending on your usage, you can also safely uncheck "Automatically connect new USB devices" and "Share Bluetooth devices with the virtual machine".
- Depending on your environment, you may also need to tweak display settings (disable 3D, enforce a screen size).
Attention:
- If you choosed the netboot install, do not forget to select your desktop environment (Ubuntu desktop or Xubuntu desktop) when reaching the Software selection screen, or you will end-up with a minimal text-only environment!
- Upon first boot, refuse to upgrade to the latest release: the whole point here is to stay in 14.04!
Upon first boot, one of the first you may want to do is install Linux guest tools:
sudo apt-get install open-vm-tools
This packet put boot-time triggers, its installation will therefore be complete only after a guest restart.
Fetch Android source code
While similar, the procedure details depends on the chosen ROM:
- For CyanogenMod, search for your device (select the vendor first) then click on the "How to build CyanogenMod" link to get instruction adapted for your device.
- For AOSP, follow the procedure which starts here.
It can be worth noting that CyanogeMod bundles in its source tree a tool allowing you to unpack boot.img files. To say it differently, CyanogenMod provides you a tool which will allow you to access the sepolicy file stored in devices and ROM archives. Google's AOSP does not provide such tool, so if you have no other imperative using CyanogenMod's source tree may be the most convenient choice, otherwise you will have to install it appart (which is quick and easy to do, so no worry here).
Here I'm following CyanogenMod 13.0 (Android 6.0) procedure. Explanation on the commands used is available on the pages linked above. Please read them, the typescript below is given only for reference purposes.
Tip: While I use apt-get in this post to stick to the lowest common denominator and keep everybody happy, you may prefer to use aptitude instead since it will take care of the dependencies in a better way (when removing a package which required the installation of some dependencies, these dependencies will be removed too, leaving your system cleaner). AFAIK the aptitude command must be installed in Ubuntu but is available by default on Xubuntu.
sudo apt-get install bison build-essential curl flex git gnupg gperf \
libesd0-dev liblz4-tool libncurses5-dev libsdl1.2-dev libwxgtk2.8-dev libxml2 \
libxml2-utils lzop maven openjdk-7-jdk pngcrush schedtool squashfs-tools \
xsltproc zip zlib1g-dev g++-multilib gcc-multilib lib32ncurses5-dev \
lib32readline-gplv2-dev lib32z1-dev
mkdir -p ~/bin
mkdir -p ~/android/system
PATH=~/bin:$PATH
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod u+x ~/bin/repo
cd ~/android/system/
git config --global user.name "Your Name"
git config --global user.email "[email protected]
repo init -u https://github.com/CyanogenMod/android.git -b cm-13.0
repo sync
# Coffee time: around 20GB are being downloaded, this may take several hours.
source ./build/envsetup.sh
breakfast
Now you have a clean and nearly complete source tree. The proprietary blobs are
missing, but you don't need them for SELinux related tasks.
Tip: Fetching the sources is a tedious process, it may be worth to do a snapshot or backup of your VM now.
Compile and install Android's SELinux toolset and libraries
Now the funny part of the trip begins ;) !
Until now the procedure should have been pretty straightforward. The goal was mainly to ensure that you have the very same environment as me. If you do, the sequel should remain straightforward too.
Under the hood Google's do not hesitate to apply deep changes to Android's source code between versions, therefore the exact compilation steps will be quite certainly version dependent (for instance AOSP master shows that the sepolicy/ directory will be moved).
I will first share my exact procedure to compile and install Android's SElinux libraries and toolset, but in order to keep the relevance of this post over time I will then add some notes about the generic approach to follow in order to solve most compilation issues.
Step-by-step procedure
Android's SELinux libraries provide the abstraction layer which will allow upper layer software to deal with Android-specific SELinux policy files. We will therefore need to compule and install them first (which, in itself, actually represents the core if the difficulties here, until you've found your way).
We will then be able to build and install SELinux tools. As we will see, fortunately these do not need to be Android specific, they only need to match the SELinux library version.
This procedure has been tested both using CyanogenMod and AOSP source code trees.
Compile and install Android SELinux libraries and first tools
First install dependances:
sudo apt-get install libapol-dev libaudit-dev libdbus-glib-1-dev libgtk2.0-dev \
libustr-dev python-dev python-networkx swig xmlto
In this post the variable $ANDROID_BUILD_TOP stores your source location (the directory where you issued the repo sync command). Feel free to change its name as you like.
ANDROID_BUILD_TOP=~/android/system
cd $ANDROID_BUILD_TOP
source ./build/envsetup.sh
By default the policy core utils compilation fails due to restorecond's Makefile being unable to locate some libraries. You have to edit this Makefile in order to use paths dynamically generated by pkg-config instead of hardcoded ones (do not confuse backticks with single quotes!):
sed -i 's/^CFLAGS ?= -g -Werror -Wall -W$/& `pkg-config --cflags --libs dbus-1 gtk+-2.0`/' \
$ANDROID_BUILD_TOP/external/selinux/policycoreutils/restorecond/Makefile
Feel free to open the Makefile with some text editor to ensure that the modification has been correctly taken into account.
And now compile and install:
cd $ANDROID_BUILD_TOP/external/bzip2/
make -f Makefile-libbz2_so
sudo make install
cd $ANDROID_BUILD_TOP/external/libcap-ng/libcap-ng-0.7/
./configure
make
sudo make install
cd $ANDROID_BUILD_TOP/external/selinux/
make -C ./libsepol/
sudo make -C /libsepol/ install
EMFLAGS=-fPIC make -C ./libselinux/
sudo make -C ./libselinux/ install
make -C ./libsemanage/
sudo make -C ./libsemanage/ install
make
sudo make install
make swigify
sudo make install-pywrap
sudo cp ./checkpolicy/test/{dispol,dismod} /usr/bin/
Attention: Do not miss the EMFLAGS=-fPIC environment variable setting when building libselinux. It will not generate any error yet, but in the next step you will be unable to build SETools. In case you missed it or did anything else wrong, simply issue a make clean and restart your compilation.
Compile and install SELinux tools
SELinux tools are provided in a prebuilt form which includes:
- Python scripts (and their shell script wrappers) within the
$ANDROID_BUILD_TOP/external/selinux/prebuilts/bin/ directory
- Python packages (including
*.o compiled files) below $ANDROID_BUILD_TOP/prebuilts/python/linux-x86/2.7.5/lib/python2.7/site-packages/.
I would have expected the source code of these tools to be available below $ANDROID_BUILD_TOP/external, but it isn't. Actually, I did not find any place where Google shared the exact version of SETools they used (FYI the GPL only mandates to share the code if it has been modified), so we will have to guess and try and do as best as we can.
The tools themselves are Python scripts, this a new evolution from SETools 4 (in SETools 3, commands like sesearch were binary executable coded in C). However, the tools themselves still show a version of 3.3.8:
$ $ANDROID_BUILD_TOP/external/selinux/prebuilts/bin/sesearch --version
3.3.8
So my guess is that Google took some early development snapshot from SETools 4. Until 4.0.0 beta SETools relied on libsepol versoin 2.4, with 4.0.0 release they started to rely on the version 2.5 of the library which is not compatible with the version of SELinux bundled in Android 6.0 (you can try to compile this, it will just fail).
So the wisest choice seems to go with SETools 4.0.0 Beta.
Install supplementary dependencies:
sudo apt-get install python-setuptools
Download and extract the source code:
cd ~/android/
wget https://github.com/TresysTechnology/setools/archive/4.0.0-beta.tar.gz
tar xzf 4.0.0-beta.tar.gz
cd ./setools-4.0.0-beta/
Due to a bug affecting Flex 2.5, we need to remove -Wredundant-decls from compiler's flags:
sed -i '/-Wredundant-decls/d' ./setup.py
And finally compile and install:
python ./setup.py build
sudo python ./setup.py install
Generic procedure (or "How to unstuck yourself")
In case the procedure above did not work in your case, here is a higher level view on how to try to progress.
There is sadly no magic (and no helper :( ) around here: the only way to get this code to compile is the classical yet dreaded cyclic "try-and-see" approach.
Try to compile a first time, it will most likely fail due to some *.h file being not found:
Search in Android's external/ directory:
find $ANDROID_BUILD_TOP/external -name filename.h
If you find the requested file, then this means that a specific version of the corresponding library or tool has been bundled within Android source code. You should therefore not try to install it from Ubuntu's package system, but instead compile and install the version bundled in Android source code.
Be aware that this goes against general advice you may found on forums: "Your compilation fails because of this library missing? Install this package then it will be fine!", by doing this you will most probably just go into worse issue: if a specific version is bundled, it is most probably because a specific version is needed (due to compatibility issues or because this version contains specific changes from Google).
BTW, if you are wondering: of course this library or tool may also have dependencies raising errors due to some *.h file being not found, and yes you should apply this very same cyclic "try-and-see" approach.
Search systemwide:
find / -name filename.h 2>/dev/null
If you find "missing" the file to be already present in your system in some standard shared library location, this mean that this dependency is probably already met in your environment but the Makefile who raised the error is too dumb to find it.
If you manually directly call this Makefile, it may be possible for you to set some environment variable fixing this (LIBDIR=/usr/lib make for instance), otherwise you may need to modify the Makefile itself (the pkg-config command may be of precious help to automatically generate missing build parameters).
Search in the packaging system:
apt-cache search filename-dev
Where filename-dev represents the name of the missing file in lowercase with the .h extension replaced by the -dev suffix (for instance, if Python.h is not found, search for python-dev). Some tweaking in the exact name may be needed to find the right package.
If you remain stuck and that even a quick search on Internet did not provide any clear answer, then apt-file will be your best friend. apt-file is not installed by default, you need to install it and generate its database:
sudo apt-get apt-file
sudo apt-file update
apt-file allows you to search for packages (even uninstalled ones) providing a particular file. To avoid having too much result, I recommend to associate it with grep as below:
apt-file search filename.h | grep -w filename.h
If there is a package in Ubuntu's repository providing this file, then apt-file should be able to find it.
Once you've found the right package, install it using apt-get install packagename where packagename is your package's name.
Tip: If you screwed something on your system, the command to reinstall a package is this one: apt-get reinstall pkg_name. It will work even when a classical remove & install would not be possible due to breaking dependencies (which is most likely for system's libraries).
Supplementary tools
At this step, you should now have a clean environment allowing you to investigate Android's SELinux rules both in compiled and source formats.
However, most chances are that at the end of your investigation you will want to take some action. In its current shape, your environment will not permit you to modify a device's sepolicy file. In fact, this file cannot be easilly replaced: it is part of the device root directory, and the content of the root directory is extracter at boot time from a RAM disk file, which in turn is stored in the device's boot image.
So you still miss two things before your environment is complete:
- A way to access and modify the device's boot image,
- A way to modify its
sepolicy file.
Fortunately, these are precisely the subject of the two last sections of this post! :)
Fetch and update device's boot image
Tools to fetch and update devices' boot image can be used for a wide variety of things apart from SELinux rules tampering. I have therefore created a dedicated answer, please refer to it.
Modify device's SELinux rules
You have two main possibilities here:
- Build a new
sepolicy file from the rules in your source tree (search for .te files to find them: find $ANDROID_BUILD_TOP -name \*.te, they are spread into several directories).
- Modify the
sepolicy file currently used by the device.
Unless you really need to build your rules from scratch, which is more a development-related task and therefore out-of-scope here, the second choice seems by far the safest one as you are sure that the only changes will be the one your explicitely made.
There has been a project to make a tool allowing you to decompile a sepolicy file into a recompilable form, allowing to freely edit rules in between. However this project has been abandonned in proof-of-concept state. You will find all information at the end of this blog post, the rest of the article contains enough details to allow anyone else interested to take over.
The currently recommended way to alter sepolicy rules goes another route: by directly modifying the sepolicy binary file. sepolicy-inject tool allows just that and is actively maintained.
For completeness sake, note that a fork of this tool exist. It adds a few features, some of them being on the original author's to-do list (like the possibility to remove a rule), don't ask me why they choosed to fork instead of contributing...
To compile and install sepolicy-inject, simply proceed as follow:
cd ~/android/
git clone https://bitbucket.org/joshua_brindle/sepolicy-inject.git
cd ./sepolicy-inject/
LIBDIR=/usr/lib make
sudo cp ./sepolicy-inject /usr/bin/
Use-case example
Let's say for instance you want to add the autorization matching the following error message:
avc: denied { read } for pid=128 comm="file-storage"
path="/data/media/0/path/to/some/file"
dev="mmcblk0p28" ino=811035 scontext=u:r:kernel:s0
tcontext=u:object_r:media_rw_data_file:s0 tclass=file permissive=0
You will need to fetch device's boot image, then unpack it to get access to it's sepolicy file.
A quick check using sesearch shows that there is indeed no allow rule (yet!):
$ sesearch -A -s kernel -t media_rw_data_file -c file -p read ./sepolicy
$
The command has no output.
Then, use the command below to add the required rule (note the similarity between sesearch and sepolicy-inject parameters):
sepolicy-inject -s kernel -t media_rw_data_file -c file -p read -P ./sepolicy
Now we can call back our sesearch command:
$ sesearch -A -s kernel -t media_rw_data_file -c file -p read ./sepolicy
allow kernel media_rw_data_file:file read;
$
sesearch output shows that the policy has correctly been updated.
You can now repack the device's boot.img file and flash it back to the device. Checking the last modification time of the /sepolicy file is an easy way to ensure that your device is now running the newly updated sepolicy file.
Conclusion
You should now have a complete environment allowing you to freely inspect and modify Android devices SELinux policies. Enjoy! :)
As a side note, there are also tools allowing to analyze and modify SELinux policy directly from the device.
