Difference between pages "HowtoCompile" and "Fakeroot vs fakeroot-ng comparison"

From Fakeroot NG
(Difference between pages)
(Move page from old installation)
 
(Move page from old installation)
 
Line 1: Line 1:
=Compiling Fakeroot-NG=
+
Some people are wondering whether fakeroot-ng is meant to kill off fakeroot. It is important to understand that fakeroot, developed by Clint Adams and Timo Savola, is an independent project that will live so long as they (or anyone else) decided to keep it alive. Trying to provide a quick answer, there are some area where it is almost impossible for fakeroot-ng to compete with fakeroot, just as there are some areas where fakeroot has no way to compete with fakeroot-ng. It can be said that fakeroot's coverage is wider, while fakeroot-ng's is deeper.
  
Here are a few tips and solutions to possible compilation problems:
+
=Design advantages of fakeroot=
 +
==Speed==
 +
Fakeroot-ng runs as a separate process. Every time the program performs a system call, the kernel performs at least two context switchs to fakeroot-ng (one before and one after the system call), even if the system call is not handled. As a result, at least under some scenarios, fakeroot-ng is considerably slower than fakeroot.
  
==Generic Compilation Instructions==
+
For example, creating a copy of the /dev folder:
In general, as the README file says, compiling fakeroot-ng is just a matter of running:
+
<source lang=bash>
./configure
+
$ time fakeroot cp -a /dev/ /tmp/
make
 
sudo make install
 
  
The project is an automake project, which means it supports out of tree builds, install targets and changing the installation path. Still, there are a few caveats that might be a problem, under certain conditions.
+
real    0m0.132s
 +
user    0m0.028s
 +
sys    0m0.076s
 +
$ rm -rf /tmp/dev
 +
$ time fakeroot-ng cp -a /dev/ /tmp/
  
==Passing Custom Compiler Flags==
+
real    0m0.434s
The following holds for any autoconf based project.
+
user    0m0.020s
 +
sys    0m0.076s
 +
</source>
  
During the ''configure'' stage, the environment is read for environment variables for CFLAGS and CXXFLAGS (a full list is available by running '''./configure --help'''). These values are then coded into the generated Makefile, and need not be set later on.
+
building fakeroot-ng:
 +
<source lang=bash>
 +
$ time fakeroot make -j2
 +
real    0m0.900s
 +
user    0m0.776s
 +
sys    0m0.284s
 +
$ time fakeroot-ng make -j2
 +
real    0m1.816s
 +
user    0m0.784s
 +
sys    0m0.384s
 +
</source>
  
For fakeroot-ng in particular, care must be taken that the ptlib library is written in C, while fakeroot-ng itself is written in C++. As a result, it is important to set any flags you want for both CFLAGS and CXXFLAGS.
+
==Platform Support==
 +
Fakeroot runs as part of the process, getting standard C interface function calls (glibc versioning trickery non-withstanding). Porting fakeroot to a new platform for an already supported operating system (say, a new CPU of Linux) is, in theory, just a matter of recompiling for the new platform. As a result, fakeroot easily spans all hardware platforms supported by, for example, Debian, including the non-Linux platforms.
  
For example, if you want to compile a version with higher optimization values, you should use the following command when running configure (assuming bourne shell or derived shell):
+
The need to know which registers are used, how system calls are performed and other parameters mean that fakeroot-ng needs to be ported to each new operating system/CPU combination individually. As a result, at the time of this writing, only Linux is supported, and there, only PowerPC and Intel's x86 and x86_64 platforms are supported. Further platforms support depends on someone familiar with the platforms writing the support code needed. The original author now has access to an ARM and a SPARC platforms, so support for these two platforms is not unlikely in the near future.
CFLAGS='-O0' CXXFLAGS="$CFLAGS" ./configure
 
  
In this particular case, it would have been shorter to just repeat the flags for the CXXFLAGS variable, but this mechanism ensures that the C and C++ compiler receive the same flags. Also note that this does not apply to preprocessor defines, which are shared between C and C++. So, if all we wanted was to disable all of the asserts in the code, we would have done:
+
Another aspect of the same problem is that, at least on Linux, the interface to the kernel is sometimes significantly different than the one defined by Posix or the Linux man pages. It is not uncommon to see a function defined in one way by the manual page, but to actually have that function translated into another by glibc, and implemented differently by the kernel. The different hooking locations mean that fakeroot has to implement the standard behavior, while fakeroot-ng needs to implement the platform dependent behavior.
CPPFLAGS='-DNDEBUG' ./configure
 
  
==Enabling Extra Warnings==
+
==Simplicity of Design==
It is possible to raise the warning level when compiling fakeroot-ng. It is written to be warning free when compiled with '''-Wall -Wextra'''. The only exception is the ''unused parameter'' warning, which I find unhelpful and counter-productive to eliminate. My debug environment is set up with the following command:
+
From the July 26, 1997 version of the fakeroot manual page:
CFLAGS='-O0 -Wall -Wextra -Wno-unused-parameter' CXXFLAGS="$CFLAGS" ./configure
+
<blockquote>'''Bugs'''<br>
 +
None so far. Be warned, though: although I've written quite a few much larger (and smaller) programs, I've never written anything that was as tiny as fakeroot, had as many bugs as fakeroot, and still was as usable as, say, fakeroot version 0.0_3, the first version that could be used to build itself.</blockquote>
  
==Configure error: Fakeroot-ng needs a compiler that supports C++11==
+
This simplicity cannot be attributed to fakeroot-ng, no matter how you stretch it. Fakeroot-ng is built as a stateful non-blocking server, attempting to use a single process to handle as many simultaneous requests from different processes as possible. It uses hacks, tricks and, occasionally, coercion to make the debugged program do what it wants. There is nothing simple about it.
As of version 0.18 of Fakeroot-ng, a compiler with a fairly complete support for the new C++11 standard is required in order to compile the program. In particular, fakeroot-ng makes use of ''decltype''[http://en.wikipedia.org/wiki/Decltype], ''nullptr''[http://www.codeguru.com/cpp/cpp/article.php/c19093/C-2011-nullptr.htm] and ''std::unordered_map''[http://www.cplusplus.com/reference/unordered_map/unordered_map/].
 
  
During the configure stage, fakeroot-ng tries to compile a small test program that utilizes all three capabilities, and checks which compiler flags are necessary in order to activate the relevant C++ language support. Mostly, this involves testing the "-std=c++11" flag, required for gcc.
+
=Fakeroot-ng advantages=
 +
Many of the advantages listed below boil down to one point - with ptrace, the control you have over the application is absolute. As such, there is nothing you can't do.
  
If no flag passes the compilation, your configure state will look something like this:
+
==Static linked executables==
 +
Because fakeroot uses the dynamic linker to wrap the system calls, executables that are statically linked, use a non-standard dynamic linker (that does not honor LD_PRELOAD), or that do not use glibc in order to access the kernel are not visible to fakeroot, and cannot be wrapped by it. While most programs are dynamically linked, a major example of a statically linked binary that is very often used is the dynamic linker itself. Fakeroot does not, and cannot, wrap the system calls performed by it. Fakeroot-ng has no problem.
  
checking for PTRACE_GETREGS... yes
+
==Fakeroot Limitations==
checking for PTRACE_PEEKUSER... yes
+
The man page for fakeroot has a section titled "Limitations". They list three major limitations. At the time of writing, fakeroot-ng has none of those limitations. It does not depend on the version of any library used, does wrap "open" and "create" in a reliable race free manner, and does not care about environment variables or any other by-product semantics, and therefor runs "configure' without a problem.
checking what flags are needed for compiler support of the ISO C++11 standard... not supported
 
configure: error: Fakeroot-ng needs a compiler that supports C++11. If you are using gcc, that
 
                  means version 4.7 or higher.
 
  
If you are using gcc with a version lower than 4.7, then your only option is to upgrade. Versions before 4.7 simply did not support the C++11 standard to the extent required by fakeroot-ng. If you are using another compiler, in particular, one that does support ''nullptr'', ''decltype'' and ''std::unordered_map'', then you might be missing the correct compilation command line. Find out what those are. As a temporary workaround, pass them through the CXXFLAGS environment variable to configure. If that works, then do contact us on the [[support|mailing list]] and we'll gladly add that detection into the configure script.
+
Fakeroot-ng does have its own limitations, but they are a result of how far the author wished to go emulating the kernel, and not any inherent limitations of the technology. At the moment, the most pressing limitation is that the dynamic linker is loaded by the kernel, and is therefor loaded from the real root of the system, and not from the chroot (if applicable). There is a plan laid out to fix this.
  
==Building a i386 binary on an AMD64/X86_64 host==
+
==Handling open and chroot==
Sometimes it is desired to build a 32bit target on a computer that, itself, is running a 64bit operating system. This is possible, but is not as straight forward as with other projects.
+
As fakeroot-ng has no problem in handling open, file ownership for unknown files is their real ownership. As a result, immediately after running fakeroot-ng, there is no change in how old files are displayed. New files are, of course, owned by the fake uid of the process (actually, the fake file system uid). Fakeroot-ng also supports chroot environments. Both are unsupported by fakeroot, due to its inability to handle "open".
  
First, make sure that your installed compiler knows how to generate 32bit binaries. On Debian based systems this requires installing the "g++-multilib" package. If you only installed the "gcc-multilib" package, then your compiler will know how to generate 32 binaries for C, but not C++.
+
Please note that there are other projects (most notable - fakechroot) that do wrap the open system call. Check out the [[fakeroot-ng vs fakechroot]] page for more details on it.
  
Once that's resolved, two things must be done. The first is to tell the build system you are cross compiling. This is done by passing ''configure'' the option '''--target=i686-pc-linux-gnu'''. This is important, as fakeroot-ng has specific code for each destination platform, and so must know which platform it is compiling for. This, however, does not actually causes make to generate 32 bit binaries (which is probably a bug in autoconf). To actually generate 32 bit binaries, the '''-m32''' flags must be passed to gcc. See the above section on passing custom compiler flags. The complete command line to compile 32 bit is:
+
==Library and Language Independence==
CFLAGS='-O2 -m32' CXXFLAGS="$CFLAGS" ./configure --target=i686-pc-linux-gnu
+
Somewhat covered above, but worth expanding. Fakeroot needs to be linked with the precise same glibc version as the one the end program is linked. If the versions don't match, the best case is that fakeroot does not work at all, and the worse case is that subtle bugs appear in unexpected places. This may not be a huge problem for open source programs compiled as part of the distribution, but when 3<sup>rd</sup> party, and especially closed source precompiled programs, are involved, this might be a major drawback.
  
===Error messages, and their meanings===
+
==Program Address Space Presence==
In each error message, the important part is displayed in bold:
+
Fakeroot is a library that gets loaded into the program's address space. This means that any further libraries that fakeroot may need to load itself are also loaded into the program's address space. Furthermore, any data structures that fakeroot might need for normal operation are subtracted from the memory available for running the actual program. Fakeroot's solution (not only due to this reason) is to "outsource" all database required to an external daemon, that performs the actual data lookup (mostly the file fake owners and type).
  
arch/linux/x86_64/platform.c:453:50: error: '''‘RIP’ undeclared (first use in this function)'''
+
Fakeroot-ng's presence inside the programs's address space, on the other hand, is minimal. The program runs while totally oblivious to the fact it is being debugged. Fakeroot-ng is written in C++, but it does not load any C++ run time library into the program's address space. The entire program address space presence is limited to a handful of memory pages, typically four, allocated explicitly by fakeroot-ng.
This means that you told the compiler to compile in 32 bit, but did not tell configure that your target is 32 bit.
 
  
arch/linux/i386/platform.c:80:44: error: '''‘ORIG_EAX’ undeclared (first use in this function'''
+
==Security==
This means that you told fakeroot-ng you are compiling for a 32 bit target, but did not tell the compiler to compile for 32 bit.
+
Security is a goal for neither fakeroot nor fakeroot-ng. Still, as far as the technology goes, there is a world of difference between the two. With fakeroot security is not a goal partly because, had it been a goal, it would be unobtainable. Bypassing the fakeroot syscall wrapping is as easy as compiling part of the program statically. As of this writing, fakeroot actually exhibits '''accidental''' breaking away of processes run by an autoconf generated configure script.
  
parent.cpp: In function ‘void init_handlers()’:
+
Fakeroot-ng, on the other hand, uses a technology that leaves the actual program no say over whether it wants to be traced. The system calls hooks are enforced by the kernel, and the data stored in the pages mapped into the process is protected by the machine's MMU from alteration. While the system may not be full proof, it can be made so if desired.
parent.cpp:210:1: error: '''‘sys_fstatat64’ was not declared in this scope'''
+
 
parent.cpp: In function ‘bool hma_state0(int, pid_t, pid_state*)’:
+
=Relative Roles=
parent.cpp:986:29: error: '''‘SYS_mmap2’ was not declared in this scope'''
+
 
parent.cpp: In function ‘bool hma_state3(int, pid_t, pid_state*)’:
+
Fakeroot-ng is not likely to kill fakeroot, nor is it likely to any time soon. Fakeroot's wider platform support and better performance means it is likely to be the preferred tool wherever it gets the job done. Fakeroot is still continuously developed, and, in any case, is much more mature than fakeroot-ng.
parent.cpp:1054:38: error: '''‘SYS_mmap2’ was not declared in this scope'''
+
 
This means that you passed the flags to compile 32 bit to CFLAGS, but not to CXXFLAGS.
+
That said, fakeroot-ng does have capabilities that fakeroot does not, and where those are needed, it makes an excellent tool.

Latest revision as of 16:58, 22 April 2019

Some people are wondering whether fakeroot-ng is meant to kill off fakeroot. It is important to understand that fakeroot, developed by Clint Adams and Timo Savola, is an independent project that will live so long as they (or anyone else) decided to keep it alive. Trying to provide a quick answer, there are some area where it is almost impossible for fakeroot-ng to compete with fakeroot, just as there are some areas where fakeroot has no way to compete with fakeroot-ng. It can be said that fakeroot's coverage is wider, while fakeroot-ng's is deeper.

Design advantages of fakeroot

Speed

Fakeroot-ng runs as a separate process. Every time the program performs a system call, the kernel performs at least two context switchs to fakeroot-ng (one before and one after the system call), even if the system call is not handled. As a result, at least under some scenarios, fakeroot-ng is considerably slower than fakeroot.

For example, creating a copy of the /dev folder:

$ time fakeroot cp -a /dev/ /tmp/

real    0m0.132s
user    0m0.028s
sys     0m0.076s
$ rm -rf /tmp/dev
$ time fakeroot-ng cp -a /dev/ /tmp/

real    0m0.434s
user    0m0.020s
sys     0m0.076s

building fakeroot-ng:

$ time fakeroot make -j2
real    0m0.900s
user    0m0.776s
sys     0m0.284s
$ time fakeroot-ng make -j2
real    0m1.816s
user    0m0.784s
sys     0m0.384s

Platform Support

Fakeroot runs as part of the process, getting standard C interface function calls (glibc versioning trickery non-withstanding). Porting fakeroot to a new platform for an already supported operating system (say, a new CPU of Linux) is, in theory, just a matter of recompiling for the new platform. As a result, fakeroot easily spans all hardware platforms supported by, for example, Debian, including the non-Linux platforms.

The need to know which registers are used, how system calls are performed and other parameters mean that fakeroot-ng needs to be ported to each new operating system/CPU combination individually. As a result, at the time of this writing, only Linux is supported, and there, only PowerPC and Intel's x86 and x86_64 platforms are supported. Further platforms support depends on someone familiar with the platforms writing the support code needed. The original author now has access to an ARM and a SPARC platforms, so support for these two platforms is not unlikely in the near future.

Another aspect of the same problem is that, at least on Linux, the interface to the kernel is sometimes significantly different than the one defined by Posix or the Linux man pages. It is not uncommon to see a function defined in one way by the manual page, but to actually have that function translated into another by glibc, and implemented differently by the kernel. The different hooking locations mean that fakeroot has to implement the standard behavior, while fakeroot-ng needs to implement the platform dependent behavior.

Simplicity of Design

From the July 26, 1997 version of the fakeroot manual page:

Bugs
None so far. Be warned, though: although I've written quite a few much larger (and smaller) programs, I've never written anything that was as tiny as fakeroot, had as many bugs as fakeroot, and still was as usable as, say, fakeroot version 0.0_3, the first version that could be used to build itself.

This simplicity cannot be attributed to fakeroot-ng, no matter how you stretch it. Fakeroot-ng is built as a stateful non-blocking server, attempting to use a single process to handle as many simultaneous requests from different processes as possible. It uses hacks, tricks and, occasionally, coercion to make the debugged program do what it wants. There is nothing simple about it.

Fakeroot-ng advantages

Many of the advantages listed below boil down to one point - with ptrace, the control you have over the application is absolute. As such, there is nothing you can't do.

Static linked executables

Because fakeroot uses the dynamic linker to wrap the system calls, executables that are statically linked, use a non-standard dynamic linker (that does not honor LD_PRELOAD), or that do not use glibc in order to access the kernel are not visible to fakeroot, and cannot be wrapped by it. While most programs are dynamically linked, a major example of a statically linked binary that is very often used is the dynamic linker itself. Fakeroot does not, and cannot, wrap the system calls performed by it. Fakeroot-ng has no problem.

Fakeroot Limitations

The man page for fakeroot has a section titled "Limitations". They list three major limitations. At the time of writing, fakeroot-ng has none of those limitations. It does not depend on the version of any library used, does wrap "open" and "create" in a reliable race free manner, and does not care about environment variables or any other by-product semantics, and therefor runs "configure' without a problem.

Fakeroot-ng does have its own limitations, but they are a result of how far the author wished to go emulating the kernel, and not any inherent limitations of the technology. At the moment, the most pressing limitation is that the dynamic linker is loaded by the kernel, and is therefor loaded from the real root of the system, and not from the chroot (if applicable). There is a plan laid out to fix this.

Handling open and chroot

As fakeroot-ng has no problem in handling open, file ownership for unknown files is their real ownership. As a result, immediately after running fakeroot-ng, there is no change in how old files are displayed. New files are, of course, owned by the fake uid of the process (actually, the fake file system uid). Fakeroot-ng also supports chroot environments. Both are unsupported by fakeroot, due to its inability to handle "open".

Please note that there are other projects (most notable - fakechroot) that do wrap the open system call. Check out the fakeroot-ng vs fakechroot page for more details on it.

Library and Language Independence

Somewhat covered above, but worth expanding. Fakeroot needs to be linked with the precise same glibc version as the one the end program is linked. If the versions don't match, the best case is that fakeroot does not work at all, and the worse case is that subtle bugs appear in unexpected places. This may not be a huge problem for open source programs compiled as part of the distribution, but when 3rd party, and especially closed source precompiled programs, are involved, this might be a major drawback.

Program Address Space Presence

Fakeroot is a library that gets loaded into the program's address space. This means that any further libraries that fakeroot may need to load itself are also loaded into the program's address space. Furthermore, any data structures that fakeroot might need for normal operation are subtracted from the memory available for running the actual program. Fakeroot's solution (not only due to this reason) is to "outsource" all database required to an external daemon, that performs the actual data lookup (mostly the file fake owners and type).

Fakeroot-ng's presence inside the programs's address space, on the other hand, is minimal. The program runs while totally oblivious to the fact it is being debugged. Fakeroot-ng is written in C++, but it does not load any C++ run time library into the program's address space. The entire program address space presence is limited to a handful of memory pages, typically four, allocated explicitly by fakeroot-ng.

Security

Security is a goal for neither fakeroot nor fakeroot-ng. Still, as far as the technology goes, there is a world of difference between the two. With fakeroot security is not a goal partly because, had it been a goal, it would be unobtainable. Bypassing the fakeroot syscall wrapping is as easy as compiling part of the program statically. As of this writing, fakeroot actually exhibits accidental breaking away of processes run by an autoconf generated configure script.

Fakeroot-ng, on the other hand, uses a technology that leaves the actual program no say over whether it wants to be traced. The system calls hooks are enforced by the kernel, and the data stored in the pages mapped into the process is protected by the machine's MMU from alteration. While the system may not be full proof, it can be made so if desired.

Relative Roles

Fakeroot-ng is not likely to kill fakeroot, nor is it likely to any time soon. Fakeroot's wider platform support and better performance means it is likely to be the preferred tool wherever it gets the job done. Fakeroot is still continuously developed, and, in any case, is much more mature than fakeroot-ng.

That said, fakeroot-ng does have capabilities that fakeroot does not, and where those are needed, it makes an excellent tool.