Ttyjfyk

My company uses a small out-dated cluster (CentOS 5.4) to do number crunching (finite element calculations to be more specific). They are using a commercial package and have no idea of Linux. They don't want to change anything on the machines as long as they run, which I accept as a time-effective policy. I do not have administrative rights.

I can have them install smaller packages, but not change e.g. the python version from 2.4 to 2.6+, so I decided to compile the current version (./configure --prefix=/home/mysuser/root2) and ran into a few problems with the dependencies (wrong version of e.g. readline, zlib, curses, bz2 ... or packages not found). I also need to update gcc which complaines about missing GMP, MPFR and MPC.

The reason for doing this is I'd like to compile other test software to do run on these machines.
What can I do to effectively install the packages I need in order to compile the software I need to work with? I'm elsewhere using archlinux and would find it quite handy to be able to do something along the lines

pacman --root /home/myuser/root2 -S <package>

But I have no idea if this is possible or clever.

Other related SE questions: gentoo-prefix and pkgsrc seem to be not so easy (I may be wrong, though).

Your management is wise in not trying to upgrade a working cluster that is performing an important function based on a proprietary package.

Backporting packages is time consuming and risky, that is, not always feasible. You might avoid the time penalty if you can finding the packages that you want to install in the original CentOS 5.4 repository or in some CentOS 5.4 backport repository. While you can have several versions of GCC on one host at the same time (the embedded systems/cross compile folks do this all the time), but it is not trivial to have more than one glibc in a single run-time environment.

So, you are best advised to work in a separate, newer environment that has the packages that you need and find some way to test the output of the old environment in the new one. In any event, do not risk breaking anything in the old environment or you may need all of the stackexchange.com reputation points that you can get to find your next job ;-)

Installing packages from distributions is often difficult when you don't have root permissions, as they assume a fixed directory layout and the dependency system tends to require some packages with setuid or setgid programs that you can't install as non-root.

Compiling from source is more often than not the easiest way. (And if you're after speed, you can choose the best compilation options for your particular processor model.)

To organize the packages that you compile (or install by extracting tarballs), I recommend using stow or the more powerful but more complex xstow. Their basic mode of operation is to install each package in a separate directory, then create symbolic links to put them all together. Here's a typical compilation and installation session with stow:

tar xzf foobar-42.tar.gz
cd foobar-42
./configure --prefix=~/software/stow/foobar-42
make
make install
cd ~/software/stow
stow foobar-42

That last command creates symbolic links from files and directories under ~/software/stow under ~/software. For example, if ~/software/stow/foobar-42 contains a lib/foobar directory and files bin/foobar and man/man1/foobar.1, then you will end up with symbolic links

~/software/bin/foobar -> ../stow/foobar-42/bin/foobar
~/software/lib/foobar -> ../stow/foobar-42/lib/foobar
~/software/man/man1/foobar.1 -> ../../stow/foobar-42/man/man1/foobar.1

To uninstall a program, run stow -D foobar-42 in the ~/software/stow directory, and delete ~/software/stow/foobar-42. To make a program temporarily unavailable (e.g. to try another version), just run the stow -D part.

See also Non-Root Package Managers ; Best way to upgrade vim/gvim to 7.3 in Ubuntu 10.04?

What is an effective method for installing up-to-date software on an out-dated production machine?

I encounter this frequently while testing a few open source projects I contribute to. For example, I need modern cURL, Git, Wget and a few other tools on Fedora 1. It is easier to use Fedora 1 to test GCC 3 rather than trying to build GCC 3 on a modern machine. I also need modern tools on modern Solaris because Solaris ransomware's bug fixes and updates. If you don't buy a support contract then you don't get the updated software.

For your CentOS machine I believe you have three options.

First, you can use a repository like Red Hat's Software Collections (SCL). I use the SCL to provide modern packages on a CentOS 7 production web server. SCL provides an updated Apache, updated Python, updated PHP, etc. Also see How to update Apache and PHP using SCL?.

There are other repos specifically for Red Hat and CentOS, like Remi Repos. I had a lot of trouble with Remi in the past on CentOS 7 so I don't use it.

Second, you can use an external package manager and additional repos (in addition to the native CentOS package manager). I don't use this method, but Linux From Scratch gives a good survey of the benefits of different package managers.

Third, you can install software locally. Here, locally means you install in /usr/local or /opt or similar. However, you also need to install dependent libraries locally, too. That's where the challenge lies.

I use local installs frequently. You can find my collection of scripts at GitHub | BuildScripts. There are a few benefits to the build scripts.

• They are easy to use - they provide consistent build settings, including Release configurations and RUNPATHs.


• They are easy to remove - rm -rf /usr/local followed by hash -r is usually all that is needed to start over.


• They download and build dependencies as required.

If interested, all you need to do to manage dependencies is ldd prog to determine the external libraries. The ones I update are the important ones, like libcurl, libssl, libcrypto, libxml, etc.

They scripts use a simple method to manage dependencies. Anything more than a week old is rebuilt. That ensures you always get the latest tools and libraries without the complex dependency tracking.

You can do some interesting things with method (3). For example, I acceptance test libraries. One of the things I do is build all the local software with -fsantize=undefined or -fsantize=address by adding the flags to CFLAGS and CXXFLAGS.

I've uncovered more bugs that I can count. See, for example, Unistring 0.9.10 and Undefined Behavior sanitizer findings.

The one downside to local installs is those god damn path problems that plague Linux. The software in /usr/bin will use the libraries in /usr/local/lib rather than /usr/lib. I've made a few systems unstable because programs in /usr/bin use the wrong libraries.

There is no way to set a policy of "binaries in /usr/bin can only use libraries in /usr/lib" (and similar for libraries depending on other libraries). The idiots who thought it was a good idea to compile and link against one library, and then load the wrong library at runtime should get a Darwin award.

Linux is the only major OS that has not managed to solve the problem. Other OSes, including AIX, BSDs, OS X and even Windows managed to solve the problem.