CentOS policy routing – why yes, it can be done!

Over the years working in LAN networking there are several situations that dictate a host/server have multiple IP addresses on the same or different, physical or logical, devices.  For instance, connecting to a private management-only network/vlan, offering connectivity to a inside network on a private NIC, etc etc.


This scenario often causes two somewhat annoying behaviours:


1) the return traffic often is sourced from the “primary” IP address of the host/server, most often the one that is on the subnet associated with the default gateway

2) a surprising number of alleged “network administrators” seem to think having multiple gateways (one for each IP address of course )  is a good idea.  Well, over the years I have come across this situation and, in every case, this has obviously  NEVER WORKED.  


Situation #2 can only be fixed by not mind numbingly entering multiple gateways without restraint.  As for situation #1, RedHat/CentOS and derivatives support via iproute2 the ability to make traffic rules, ensuring that IP traffic is sourced by a particular IP in cases you can define.  Great!  Multiple NIC or logical interface routing on Linux is possible! (and yes, it involves having multiple gateways, but not stupidly and blindly adding them in the routing table….)

It is very simple to implement and involves the steps below.  As an example, lets assume we created a management VLAN (VLAN4) and want to add an logical interface on a server in that VLAN to access it internally.  We will be using as an inside network.


Step 1: Create a VLAN interface

This creates the necessary interface on VLAN4 from primary physical interface eth0:

vi /etc/sysconfig/network-scripts/ifcfg-eth0.4


Step 2: Create a iproute2 table for that management network

Edit /etc/iproute2/rt_tables to add a new entry and give it a arbitrary (unused 😉 ) name:

vi /etc/iproute2/rt_tables

# reserved values
255     local
254     main
253     default
0       unspec
# local
#1      inr.ruhep
200     MGMT

Note that between 200 and MGMT is a tab character.


Step 3: Create a default route for that network

vi /etc/sysconfig/network-scripts/route-eth0.4

default table MGMT via

this creates a default route for the MGMT/ network to which is your inside routing intelligence.

Step 4: Create routing rule for

To ensure that traffic received on is utilizing the MGMT network only as a source address, a rule must be defined to enable this:

vi /etc/sysconfig/network-scripts/rule-eth0.4

from table MGMT


and thats it!  restart your network:

/etc/rc.d/init.d/network restart


Using iproute2 commands, we can check that what we did works (as well as using wireshark 😉 )


[root@server network-scripts]# ip rule show
0:      from all lookup local
32765:  from lookup MGMT
32766:  from all lookup main
32767:  from all lookup default
[root@server network-scripts]# ip route show dev eth0.4  proto kernel  scope link  src dev eth0  proto kernel  scope link  src dev eth0  scope link  metric 1002 dev eth0.4  scope link  metric 1006
default via dev eth0


Note: this also would work with a second physical interface, for instance to utilize a second NIC card instead of a VLAN logical interface, substitute all use of eth0.4 for eth1.


CentOS 6.3 released – features….and bugs :)

CentOS logo


Well folks its finally here ; http://lists.centos.org/pipermail/centos-announce/2012-July/018706.html

Among the changes, imitating its parent RedHat6.3, OpenOffice is replaced by libreoffice, I have yet to try it out but apparently has a bit more community support and can read .docx files.   They are also deprecating matahari, a management API I know little about, in favor of another suite (CIM).  RedHat even recommends matahari removal, which is fine by me since I never bothered to learn it 🙂

Bind9.7 is also replaced by Bind9.8 (yea!), a rare move by RedHat in my memory since they dislike changing version numbers in the lifecycle of each distribution.

Another notable addition to CentOS6.3 is the tools to convert physical to virtual machines for use with KVM, virt-p2v and virt-v2v (for migrationg a virtual-to-virtual installation).  Unlike RedHat6.3 that ships an .ISO image seperately to boot from in order to use these tools, CentOS includes that .ISO in the .rpm.   I look forward to trying it out.

I have already upgraded a number of guests (my KVM host however is still running 6.2, I have not worked up the courage to give it a go yet, it should go well but I can’t be bothered with glitches right about now), and all seems to work smoothly; I did come across one issue however, if you utilize IPv6 resolvers, a new(?) bug in libresolv causes a segmentation fault crash in applications that make a particular call to it;  sendmail, freshclam/clamav, emacs, openvpn, chrony, postfix.  Here is the Redhat bugzilla entry, as well as the CentOS entry.

Even if only one of your resolvers is an IPv6 address, affected software will crash;


freshclam[6598]: segfault at 1 ip 00007f9be8b37596 sp 00007fff9ffac0b0
 error 6 in libresolv-2.12.so[7f9be8b2b000+16000]

sendmail[7374]: segfault at 1 ip 00007f95d7b73596 sp 00007fffa93295a0 
error 6 in libresolv-2.12.so[7f95d7b67000+16000]


I look forward to the fix.  In the meantime I will get to tinker with P2V/V2V tools.