Panix V-Colo VPS: Frequently Asked Questions

Panix V-Colo Frequently Asked Questions

Words or phrases marked with this icon

will open a helpful Wikipedia article.

Answers

Panix and Xen

What is Xen? What is a Hypervisor or a Virtual Machine Monitor?
A Hypervisor , or Virtual Machine Monitor, is software that makes one computer seem to be many "virtual machines", each one capable of running an operating system that functions entirely independently of any other. Each OS is completely isolated from every other, and each one thinks it has sole control over all the resources of the machine. With this isolation comes security- even if one of the virtuals is compromised, the others are unaffected.
Xen is an Open-Source hypervisor with very widespread industry acceptance.
Links:
- Xen Project site
Why choose Xen over other options?
Xen has much better performance than other virtualization products- as little as one-tenth the overhead. This makes it cheaper to run, but it also makes every virtual perform better on I/O operations- that is, using disk storage or the network. Xen is now becoming the de facto industry standard for virtualization, with the support of almost all of the major industry players, including Intel, IBM, Sun, AMD, Cisco, Dell, HP, Novell, and Red Hat, among others.
Xen is Open Source Software, so it benefits from a rapid development pace and good security.
Does Panix contribute to the Xen project?
Panix has a NetBSD developer on staff. We've contributed significantly to the NetBSD Xen port.
What kind of support do you provide?
Please see our support page.
Why don't you offer (X, Y, Z) distributions?
We have to make sure that any distribution we offer has been tested so that we know it's stable. Only reasonably modern kernels will work with Xen, so some older ones will not be available. We already support a number of distributions of Linux, as well as FreeBSD, NetBSD, and OpenBSD. If we don't list what you're looking for, please ask - we can often add support for distributions within a few days.
What kind of hardware is Panix using for the V-Colo service?
Some Legacy hardware differs slightly from the hardware listed here, but not in any significant way.
- CPUs - Servers are Intel Xeon or AMD Epyc multiprocessor rackmount units. As of 2022 most are configured with two CPUs (12-24 cores) and 256GB RAM.
- Ceph Distributed Storage - Virtual disks are hosted on a Ceph cluster. Each block of data is replicated across three different servers making it fault-tolerant. Devices used to hold the virtual disks are a hybrid of Solid State Drives (SSDs) and 10K RPM disks.
  Since the virtual disks are replicated across multiple servers, a server hardware failure does not require lengthy downtime in order to dispatch an admin to the server location, remove disk drives, put them in another server, and boot it up. Instead, we can immediately move the V-Colos to another server.
- Internet connection - As of 2022, our V-Colo servers connect to the Internet primarily through both Cogent and 365 Datacenters. Other carriers may be used in unusual circumstances and this may change over time, but we will always maintain at least two separate connections "upstream", each with enough bandwidth to support 100% of our V-Colo customer usage.
- Physical infrastructure - Panix V-Colos are currently available in a Colocation center in lower Manhattan. Power is backed up by dual generators and UPSs.
- Rsync Backup server - Our backup server is a server with many large low-cost disks (we don't use Ceph for our backup server). It backs up all Panix servers, and now V-Colos as well. Tapes (actually, large disks) are made from this server, for Panix hosts, but not for V-Colos unless customers specifically arrange for this, for an additional charge. Even without taping, however, customer files are always backed up on the rsync backup server, and can be retrieved in case of disaster.
Don't you know it's spelled "Collocate" and not "Colocate"?
Well, that's what the dictionary says today. But dictionaries are made by those who spell, or something like that. We like it better with one "l", and so does about 33% of the rest of the world (based on Google hits... and it was 20% when we first wrote this a year ago). Think of it as the cutting edge of spelling technology. In ten years, most people will spell it our way.
"Spelling technology"? You can't be serious!
Semiserious. As you may have noticed, we don't put punctuation inside quotation marks either, unless it's part of the quotation. Shall we get back to V-Colos now?

Practical matters

Can I upgrade to a bigger virtual machine later on?
Yes. You can change services, and increase any add-ons you may have, at any time. You can also decrease your services or add-ons at any time, unless you are pre-paying yearly. (You can't downgrade prepaid services.)
Almost any change can be made through our web control panel.
- Log into the control panel at config.panix.com/vc.
- Click on the "V-Colo Management" tab.
- Click on the "Service Options" tab, and the "Service Type" dropdown menu item.
Changes will take effect immediately, though you may need to reboot your server to make use of increased resources.
You can add extra disk space as an additional virtual disk, mounted on a new mount point, or you can expand your current disk. If you expand your current disk, you'll need to expand the filesystem as well using whatever tools are appropriate for your OS and filesystem.

Try out our web control panel here, using username "demo" and password "demo".) Service decreases also take effect immediately, but billing at the lower rate starts at the beginning of your next billing period.
I upgraded my disk space (or Panix gave everyone more free disk space), but my disks are still the same size. How do I access the additional space?
If you upgrade your server, or we increase the amount of disk that comes with your class of V-Colo, you have access to more GB of disk space, but you still have to add that space to an existing virtual disk, or create a new virtual disk. You can do this in the Control Panel's Disks tab.

Once you create a new disk, or resize an existing one, you need to make sure that your OS handles it correctly. For a new disk, make sure it's being mounted. If it's an old disk, you'll likely need to use an OS utility to resize the partition on the disk (since the disk is now bigger). This utility varies depending on the OS and filesystem you're using.
How do I figure out how much disk space my distribution will need?
RAM and disk space usage are basically identical to what they'd be if you were running an actual (not virtual) server. The space taken by the various OSes and distributions we support is listed here, along with brief descriptions of what's included with each install. Of course, you can add or remove anything you like to your server- a truly stripped-down system might run in just a few hundred MB or less.
How can I increase my disk space or RAM, get more IP addresses, or get additional transfers? Can I reduce my service level or add-ons?
You can log into the web control panel to modify your settings, which become active upon reboot. (You can try it now, using username "demo" and password "demo".)

You can also reduce your service level or add-ons at any time. However, the change will not be reflected in your bill until the beginning of the next billing cycle.
Note: If you are reducing your disk space, either by giving up additional disk blocks or by reducing your level of service, you must first ensure that you have enough disk space to continue.
What's a GiB? How big are your GB? What about MiB and MB?
Short answer: We always use the larger number, so you're getting more for your money.
Long answer: The true meaning of "Gigabyte" ("GB" for short), as defined by international standards-setting organizations, is 1000 * 1000 * 1000 bytes (exactly one billion bytes). However, in the USA, when people speak of a "Gigabyte", they normally mean 1024 * 1024 * 1024 bytes, or 1,073,741,824 bytes. So when you see people talk about (for example) a Gigabyte of RAM, they always mean 1,073,741,824 bytes. Similarly, a Megabyte ("MB") is defined as 1,000,000 bytes, but most people mean 1024 * 1024 = 1,048,576 bytes.

The one exception to this rule is the disk drive industry. Disk manufacturers always use the "true" meaning of the words "Gigabyte" and "Megabyte". So a 100GB disk will hold about 100,000,000,000 bytes, and not 107,374,182,400 bytes.

On the Panix V-Colo pages, all references to "GB" or "Gigabytes" refer to the larger (1073741824) value, including virtual disk drive sizes. So, for example, a 3 GB configuration includes 3,221,225,472 bytes of storage, not 3,000,000,000. Similary, our "MB" always mean 1,048,576 bytes.

"GiB" is the short form for "Gibabyte". This is the correct term for the larger value (1073741824) above. "MiB" is short for "Mibabyte", the correct term for what we're calling "MB". However, almost nobody in the USA uses "GiB" and "MiB", and many outside the USA don't use it either.

To avoid confusing those who are not steeped in the arcana of standardized weights and measures (that is, just about everybody), we're using the popular "GB" instead of "GiB", and "MB" instead of "MiB", even though "GB" and "MB" are technically incorrect, throughout our site.
How exactly do you measure transfers used? Is there any limit on how many transfers I can do?
To measure your use of transfers, we use "flow accounting" on the routers that handle all V-Colo traffic. The size of every packet sent to or from your V-Colo, except for traffic to the rsync backup server, is summed every
fifteen minutes and added to your totals for the month. Whichever total (sent or received) is larger is your total transfer usage. At the beginnning of every billing period, we reset your totals to zero.

Traffic between V-Colos is free and does not show up in your usage report (just like traffic to the backup server, if any). However, if you have multiple V-Colos, and have requested physical diversity (putting them in different data centers), then traffic between your V-Colos that crosses between data centers will count as transfers. We will not put your V-Colos in different data centers unless you *explicitly* ask us to, so you don't need to think about this unless you've done so.

There is a limit of 100Mbps on any transfers to or from your server. Practically speaking, you are unlikely to ever be in a situation where you can see this limit, since all transfers over the Internet are likely to be slower than that, unless you have a very fast machine with a very fast connection somewhere else on the net- much faster than a home DSL or Cable connection.

You are also limited by the transfers you purchase. If you use more than the amount you paid for, and then use up all the excess transfers you authorized us to bill you for, your connection to the net will be shut down until the start of the next billing cycle (or until you authorize an increase in the excess transfers). More details are available in this FAQ.
How can I get an idea of how much of my transfer allocation I've used so far?
Go to the Transfers tab in the Web Control Panel; select your V-Colo if you have more than one. Transfers used are displayed as a graph and in numbers. The 100% total on the graph is the sum of the transfers included in your V-Colo plan plus those that you've prepaid, whether at the standard rate or at the cheaper bulk rate. If you've set a limit above $0 in the "Extra transfers allowed" box (in the Addons section of the Control Panel), the transfers that you might incur for that additional fee are not included. Therefore if you've set the limit above $0, you can run over 100%.
What happens if my server does more data transfers than I've paid for?
Note that over 98% of our customers never reach the limit of transfers included in their level of service, and thus never need to pay extra. This is for those to whom it will matter:

Each V-Colo package includes a specified transfer quota (in GB/month). Additionally, you can enlarge that monthly transfer quota buy buying standard blocks ($5/4096 GByte) of transfers. Your total transfer quota is what your package includes plus whatever transfer blocks you've purchased. This is the amount represented by 100% on the transfer bar in the V-Colo Management section of the Control Panel.

Any transfers beyond your total transfer quota are extra. In the "Addons" section of the Control Panel, you have entered a dollar amount (between $0 and $50) that you're willing to pay for transfers above your total transfer quota. We convert that dollar amount to GB at the standard rate ($0.05/GByte), and disable network access for your V-Colo when you exceed that limit.

So the complete answer is this: If, in a given month, your transfers exceed your total transfer quota, you can continue to transfer data, and accumulate transfer charges, until you've reached the limit you specified in the "Extra transfers allowed" box.
How do I limit how much data my server can transfer?
There is always a limit, though virtually nobody ever reaches it. For most customers, that limit is the same as their transfer quota - whatever comes with their VC package, plus any extra transfer blocks purchased. However, if you want to allow for the possibility of extra transfers, you can tell us to permit more transfers than you've paid for. Any such transfers will incur a charge. To do this, go to the Addons tab in the Web Control Panel and select your V-Colo if you have more than one. You'll see that you can specify a dollar amount that you're willing to pay for transfers in excess of your total transfer quota. We convert that amount to gigabytes at the standard rate ($0.05/GByte) and allow that much over your allocation. If your transfers exceed even that amount, network access for your V-Colo will be disabled (though you can still access it via the console) until the start of the next billing period. The amount you enter can be anything from $0.00 to $50. Don't forget, it's a LOT cheaper to buy transfer blocks up front than to pay by the GB.
Why should I buy additional transfers at the regular rate instead of just letting you bill me for them if I use them?
If you're one of the rare customers who actually needs extra transfers, buying some ahead of time gives you the option of tighter control over your budgeting. Also, the transfer graph will include your prepurchased transfers in its display.
You can purchase additional blocks of transfers for $5/4096 GByte.
I'm having a big month for transfers. Can I buy a bulk block in the middle of the month?
You can purchase more bulk transfers for the current billing period at any time up through the the end of the period. However, once the period is over, you can not retroactively purchase transfers at the bulk rate.
If I use the free rsync backup service, won't that use up all of my transfers?
No. Our software specifically excludes rsync traffic from its tally of your network usage. Transfers to our rsync server are free.
How do you charge for services- before they're used, or after?
We charge (or bill you) in advance for fixed-fee services. That includes everything except for excess transfer fees. In this, we're just like your phone or cable companies (except that we don't suck)- typically, they also bill you ahead of time, for the service they're about to deliver, for all fixed-cost services.

When you incur excess transfer charges, we charge (or bill) you for them on your next bill. Again, this is just like your phone or cable companies- they may bill in advance for basic service, but they bill on the next statement for long distance, or pay-per-view, or other services that incur charges only when used.

For now, the only service we have that can accumulate charges is excess transfers. If you want to make sure that you never get billed for such charges (perhaps because you have a strict budget), or if you want to limit them to a certain maximum amount, you can do so. See this FAQ entry for info on how to do that.
How do I close my account?
You can do this from the Web Control Panel, or you can We'll ask you to authenticate over the phone using the challenge and response you set up with your account. (Please don't send them in email. That's not secure.)

On the technical side

How do I reboot my server?
You have three options:
- You can reboot your machine from our web control panel using this link.
- You can log into your machine normally (if it's running normally) and use the normal commands to reboot.
- You can reboot your machine from its console, if it's running (even in single-user mode), by logging in through our console server. See the Console Access page (which you can bookmark, or reach from the Web Console link in the left sidebar).
How do I boot into single-user mode?
If you're using NetBSD-9 or earlier or a legacy Linux kernel provided by Panix, you can boot into single-user mode using the web console .

For all other systems including most Linux, this needs to be done on the console at the boot loader prompts. The procedure varies between systems and even between different versions of the same system. You'll need to consult the vendor documentation for this information.
1. Reboot the system either from within your V-Colo or on the web console .
2. Connect to your console .
3. When the boot loader screen appears interrupt the countdown
4. Follow the instructions provided by the vendor for the version of the operating system installed on your V-Colo
How do I log into my console? How is the console different from a regular login?
You can log in to your server using ssh. But the console is different- it's like a keyboard and screen physically attached to your server. Even if the network is down (perhaps because you've reached the limit you set on transfers for the current billing cycle), you can still attach to your console, which you can reach from the Web Console page.
How do I get superuser access on my system?
Use the root password and log in as root, or use "su" or "sudo" after logging in with a non-privileged account.
How do I change my root password? What do I do if I lose/forget my root password?
If you know the password, you can change it just as you would on a non-virtual server (using "passwd", in most cases).

If you don't know your password, connect to our console server and reboot the system in single-user mode (you can also reboot it from our Control Panel). Then use the console server to connect to your V-Colo's console, log in (without a password, since it's in single-user mode), and set a new password using the usual tool (typically "passwd").
For Linux, if your O/S doesn't provide a single-user mode or provides one which requires a password, you may need to use an alternate recovery procedure.
What advice can you give me on securing my V-Colo?
You can block offending hosts with the firewall system within your V-Colo.
We recommend installing the fail2ban package, which can be configured to automatically disable traffic from hosts that are attempting to break in. It can easily monitor ssh, ftp, and most other daemons that accept logins. Any site that tries to break in will be barred after making too many failed logins within a configurable interval; fail2ban expires the ban entries after that interval so if you somehow accidentally trip it you won't be locked out forever.
How do I install a new distribution on my virtual server?
You can do this from the Web Console.
How can I find out what versions of software will be installed on my virtual, with each of the distributions?
In general, we provide the version that comes with each distribution. It is possible that we may provide a few specific updated/patched binaries to deal with severe security problems in the vanilla distribution. However, in general, we do not tamper with the contents of the distributions, so you'd be well advised to update your distribution as appropriate.

Over time, we will fill in more and more detail on our Operating Systems page.
How do I take advantage of the nightly rsync backup service? How do I recover rsynced files?
It's set up for you automatically when we first install your server. You just have to remember not to break it by damaging the configuration we've set up. That configuration exists in these two files:
```
       /etc/rsyncd.conf
       /etc/rsyncd.secrets
```
In addition, the rsync server must be enabled and running.
If you do manage to mess it up by accident, let us know and we'll help you fix it.

You may want to disable rsyncing temporarily (for example, if you want to prevent further backup runs, until we recover a file for you that was backed up previously). The easiest way to do that is to be root and then type:
```
	mv /etc/rsyncd.secrets /etc/rsyncd.secrets.saved
```
When you're ready to reenable rsyncing, just undo it:
```
	mv /etc/rsyncd.secrets.saved /etc/rsyncd.secrets
```
To restore files from the backup server to your V-Colo, you can use this command line (where NNN is the number of your V-Colo):
rsync -a rsync://vcNNN@backup.vc.panix.com/vcNNN/path/of/files/wanted/ /path/to/ restore/directory/
Though this may appear as two lines on this web page, make sure you type it in as one line.
The file /etc/rsyncd.secrets on your V-Colo contains the password for which you will be prompted.

Using a URL, as specified above, will cause rsync to use direct TCP connections on port 873, so you'll want to make sure that your V-Colo's firewall allows connections to and from backup.vc.panix.com on port 873.
What happens if I don't use the rsync backup service?
Nothing. But if something catastrophic happens to both halves of the mirror your data is on, and you're not using this service, there will be NO BACKUP of your data anywhere on our network. For some people, that's just fine. Make sure that you're one of those people before you choose not to use our backup service.

Note that our tape backup service makes tapes from our rsync server, so you MUST leave the rsync service running if you want to make use of our tape backup service.
How can I tell when the last rsync backup was made?
Log in to config.panix.com, then navigate to V-Colo Management -> VPS Info -> Status. The line "Last rsync backup" contains the date and time of the last successful backup.
How many generations of backups are available for recovery?
By default, just one. If you need to archive your data over time, you'll need to make other arrangements for this. One option would be to purchase tape backup service from us. Since we tape from our rsync backup server, you'll need to allow us to rsync your data in order to be able to tape it. You can order tape backup for your V-Colo through our Web Control Panel. Before you do, make sure you understand how many generations of backups we keep, and that that's sufficient for your needs.
How do tape backups work? What hardware is being used?
We call it a "tape backup system" because we treat it like one, and it works the same way, but we use removable hard disks instead of tapes (as do many other sites, these days). This provides much greater reliability and a much shorter MTTR (mean time to recover the data), even though it costs us a little more than tape would.

As described above, all customers who don't disable rsync are backed up nightly to our rsync server. Then, for customers who are paying for backups, we use the Amanda backup system to back up their files every night to removable hard disks. We currently keep two to three months of backups. We don't keep long-term archives of customer data (longer than three months), but we can do that for you on a custom service basis - contact us directly to arrange this.
How do I get data back from the backup service?
This tells you how to get backups from the most recent nightly rsync. If you need older data restored (or your entire V-Colo restored) from our tapes, just ask us.
How do I restore my host if it gets irretrievably corrupted?
If you really can't fix it, we may be able to restore it from our rsync backups (or from our tape service, if you're paying for it). But if even that won't work (perhaps because the damage was done long before it was discovered, and the backups contain the broken files too), you can always start from scratch. Pick an OS distribution you want installed on your server, from the Web Console.
Does Panix block any ports or kinds of traffic?
Panix currently blocks external NTP traffic to V-Colos due to security concerns. You may use the panix NTP servers instead. These are
- ntp.panix.com
- ntp2.panix.com
If you have a specific need which cannot be met by using these NTP servers, we will consider opening the port for your V-Colo. Please contact Panix staff to discuss.

Panix blocks outgoing traffic to port 25 (inbound SMTP port) by default. We routinely override this block for non-spam uses; you can request an override. You can also relay mail through our mail server without prior arrangement, at mail.vc.panix.com.

Panix may also block other ports in response to emergencies, such as an ongoing internet virus or worm. This has, to the best of our recollection, never happened so far.
Does my virtual server need a domain name? If it has one, can I have a reasonable reverse DNS entry?
It doesn't need a domain name, but you can register one if you prefer. If you don't, it'll have a name like "vc293.vc.panix.com".

Its reverse DNS will also look up to such a name by default. However, you can use the Web Console to change it.
How do I point a domain at my V-Colo?
Whether you're hosting DNS for your domain here or elsewhere, you'll need to create an A Record pointing to the IP address of the V-Colo.

You may host DNS with us for any number of domains at no additional cost. To add a DNS record, use the "Create new DNS service" option on the Order Services page. You will then be able to edit your domain's local DNS record with the DNS control panel.

You will need to update the DNS record at your domain's registrar to point to the following nameservers:
- ns1.access.net (198.7.0.1)
- ns2.access.net (198.7.0.2)
What kind of performance can I expect from my V-Colo server? How are customers protected from other customers using too many resources?
Performance analysis for V-Colos can be broken down into two major areas: CPU and Disk I/O.

CPU performance is guaranteed to be no less than 120MHz for every GB of RAM in your V-Colo. So if you order a system that comes with 4 GB RAM, you will have the performance of at least 480MHz. In fact, most of the time, most V-Colos are not using all of their CPU allocation, and any spare CPU is shared between those V-Colos that want it, again in proportion to their RAM size. So most of the time, you'll have much more than your minimum CPU allocation available to you. This is all accomplished by Xen, which allows us to guarantee CPU time, and assign proportions of CPU time, in whatever manner we choose.

Disk I/O performance is a much trickier issue. In rare cases, it is possible for a V-Colo to generate an enormous disk load that will dominate disk I/O, to the detriment of other V-Colo users. We are taking two approaches to solve this problem:
- We're throwing resources at the problem, by using SSDs along with 10K RPM disks, instead of the low-cost and much lower-performance SATA disks that some other VPS providers use.
- We're planning on adding a disk I/O scheduler which will guarantee a reasonable minimum of I/Os per second to every V-Colo. However, our operational experience has been that there is little need for this with our current architecture, so for now this has a low priority.
I've noticed the clock on my server is off a little bit. What's wrong, and what can I do about it?
Most of the time, your clock will be right. However, there are some things that can make it wrong. For example, we may "suspend" your virtual to migrate it to different hardware (we'd only do this in case of impending hardware failure, or for other good administrative reasons), or for scheduled maintenance. When we resume your OS, your system would have lost some time. There may also be bugs in Xen, or your operating system, that could conceivably cause the clock to be wrong, though this would be very unusual.
There are several solutions to this. The simplest one is to run "ntpdate" out of cron every hour (or day, depending on how much you care about timekeeping). Alternatively, you can run ntpd. However, there's a problem- ntpd will die if if sees time of out of sync by more than 1000 seconds (by design). The "-g" flag will prevent this, but just once. So, the safest thing to do would be to use a script that did this:
```
 while : ; do
                ntpdate {options ...}
                ntpd {options...}
        done 
```
Start that script when your OS starts (from rc.d, rc.local, or whatever you like).
You may find that this happens rarely enough that you don't care. When you do notice it, you can always run "ntpdate" by hand, or even reboot.
I just bought a new IP address. How do I get the v-colo to see it?
Add the ip to the appropriate (OS specific) config files. Then either use ifconfig or ip to bring up the new address, or reboot.

Panix V-Colo Frequently Asked Questions

Panix and Xen

Practical Matters

On the technical side

Answers

Panix and Xen

Practical matters

On the technical side