Creating Linux Virtual Servers

Creating Linux Virtual Servers

Wensong Zhang, Shiyao Jin, Quanyuan Wu
National Laboratory for Parallel & Distributed Processing
Changsha, Hunan 410073, China

Joseph Mack
Lockheed Martin
National Environmental Supercomputer Center
Raleigh, NC, USA

0. Introduction

Linux Virtual Server project.

The virtual server = director + real_servers

multiple servers appear as one single fast server

 Client/Server relationship preserved


Patch to kernel-2.0.36 (2.2.x in the works)

 ippfvsadm (like ipfwadm) adds/removes servers/services from virtual server. Used in

ippfvs = "IP Port Forwarding & Virtual Server" (from the name for Steven Clarks' Port Forwarding codes)

code based on

single port services (eg in /etc/services, inetd.conf)


protocols If the service listens on a single port, then LVS is ready for it.

additional code required for - IP:port sent as data, two connections, callbacks.

ftp requires 2 ports (20,21) - code already in the LVS.

Load Balancing

The load on the servers is balanced by the director using GPL, released May 98 (GPL)

1. Credits


High Availability

2. LVS Server Farm

Figure 1: Architecture of a generic virtual server

The director inspects the incoming packet - The default table size is 2^12 connections (can be increased).

Gotchas (for settting up, testing)

3. Related Works

Existing request dispatching techniques can be classified into the following categories:

3. Director Code

kernel compile options: Director communicates with real servers by one of

Figure 1: Architecture of a generic virtual server

4.1. VS-NAT - Virtual Server via NAT

popular technique for allowing access to another network by many machines using only one IP on that network. Multiple computers at home linked to internet by ppp connection. Whole company on private IP's linked through single connection to internet.

VS-NAT: Diagnostic Features

VS-NAT Example

ippfvsadm setup

 ippfvsadm -A -t -R
ippfvsadm -A -t -R -w 2
ippfvsadm -A -t -R

Rules written by ippfvsadm

Protocol Virtual IP Address Port Real IP Address Port Weight
TCP 80 80 1 8000 2
TCP 21 21 1
Example: request to

 Request is made to IP:port on outside of Director

load balancer chooses real server (here, updates VS-NAT table, then
packet source dest
inbound rewriting
reply to load balancer
outbound rewriting

VS-NAT Advantages

VS-NAT Disadvantages

4. VS-TUN - Virtual Server via IP Tunneling

Normal IP tunneling (IP encapsulation)

Tunnelling used


For ftp, http, scalability

VS-TUN Diagnostic features

Figure 4: Architecture of a virtual server via IP tunneling

Routing Table


 link to tunnel

 /sbin/ifconfig eth0:0 netmask broadcast up
route add -host dev eth0:0

ippfvsadm setup (one line for each server:service)

ippfvsadm -A -t -R
ippfvsadm -A -t -R
ippfvsadm -A -t -R


 ifconfig tunl0 netmask broadcast
route add -host dev tunl0
packet source dest data
request from client GET /index.html
ippfvsadm table is src, dest, director looks up routing table, makes src, encapsulates source, dest, GET /index.html
packet of type IPIP, server decapsulates, forwards to GET /index.html
reply from (routed via ‹html›...‹/html›

VS-TUN Advantages

VS-TUN Disadvantages

VS-DR Direct Routing

Based on IBM's NetDispatcher

 Setup uses same IPs as VS-TUN example on a local network, with lo:0 device replacing tunl device

 lo:0 doesn't reply to arp (except Linux-2.2.x).

 Director has eth0:x, servers lo:0

 When sending packets to server, just changes the MAC address for the packet


VS-DR Advantages over VS-TUN

VS-DR Disadvantages

5. Comparison, VS_NAT, VS-TUN, VS-DR

property/LVS type VS-NAT VS-TUN VS-DR
OS any must tunnel (Linux) any 
server mods none tunl no arp (Linux-2.2.x not OK) lo no arp (Linux-2.2.x not OK)
server network private (remote or local) on internet (remote or local) local
return packet rate/scalability low(10) high(100's?) high(100's?)

6. Local Node

Director can serve too. Useful when only have a small number of servers.

 On director, setup httpd to listen to (as with the servers)

 ippfvs -A -t -R

7. High Availability

What if a server fails?

Server failure protected by mon. mon scripts for server failure on LVS website.

8. To Do

9. Conclusion