CF-cards and partitioning

[tsmalmbe]

Hello all,

I misunderstood things a bit. So now I've got a CF-Card IDE-adapter and a 16Gig CF-card. Can this be partitioned somehow? Used as webcache? Or should I get another 4Gig card, install RAQCOP on that, and run this as a secondary card (as webcache or anything else)? Any suggestions?

[Davesworld]

Any card over 512MB is a waste since /var/log runs on a mounted ramdisk in flash versions and the partition that the saved logs are backed up as tarballs to is 30MB on all flash size installs. Any proxy cache you do run on a flash install is in the mounted ramdisk and NOT saved on shut down. You never want to run a proxy cache directly on a flash card partition unless you are trying to kill the flash card in short order. They simply cannot tolerate the read/write cycles of a spinning disk.

Most of us only use the proxy for filtering these days, not caching. I went all out in my early IPCop days but after months I realized that the internet is just too dynamic and I was only averaging under 10% cache hits per hour, not worth it. Sure, the script could be rewritten to accomodate the larger than 4GB Flash sizes but you will only end up with a huge / root partition that is only occupied by 110MB, not sure what you will do with the other 15GB sitting there. You simply do not need over 512MB unless you are doing heavy development work. Most people buy 2GB and 4GB only because that is all that is available to them locally, not because they need it.

The bottom line here is that if you need a large proxy cache and will really use it and need it available after reboots, do NOT use a flash install but rather a disk install. If you have a good amount of ram, you can get away with a sizable cache on a flash install but it is volatile. I actually like the fact that graphs and logs run in a ram disk on a flash install and are backed up once an hour only to the tarball in /var/log_compressed. Right install for the right needs, it's very individual.

[weizen_42]

Also with increasing CF card sizes the 'correct' size becomes increasingly difficult. The variation in real size of 4 GB cards is anything but small.
The mkflash script allows for a safe margin, but this margin becomes fairly large for 2 GB and 4 GB already.

[tsmalmbe]

OK, I will use the big-ass card for something else . What ever that is, I don't know. The log-thing is interesting though: I would surely like the logs to be in a readable format for longer than 1 hour, including graphs and stats. Can this be configured, even on a 2G or 4G card, there should be pretty plenty of space for logs?

[tsmalmbe]

Oh, another note: If I plan to install additional features from IPCOP onto my RAQCOP, how can I estimate the amount of diskspace I need? There are lots of interesting addons, but will they for instance fit in 4GB (they should)?

[Davesworld]

The whole point of why frequent writes are done to a ramdisk rather than a physical partition is CF life. The logs back up once an hour to compressed tarballs and also are compressed and saved on proper shutdown. To have a real /var/log partition on a flash drive is asking for failure. Most are only rated at a small number of read write cycles which is a tiny percentage of what a spinning platter drive is rated at.

I highly recommend having logs sent to a syslog server on another machine if you want to keep them forever or just don't use a flash installation. Flash installs are great for some but not for all situations. Acres of logs and massive amounts of proxy cache are two good reasons to stay with a disk. Myself, I have no need for more than a week's worth of logs and I love the speed of the ramdisk for things like graphs and logs, multiple times faster than any physical drive. I also do not proxy transparently as I listen to a lot of Shoutcast streams and it ruins the audio with chirping sounds because Squid can't deal with the metatagging and thinks it's supposed to drop to http 0.9 or something because it does not know what icy means. I also love the low power draw and low temp of the flash card, especially on a device that only has a 60 watt power supply. I prefer to save the power for peripherals and not waste it on the drive.

[tsmalmbe]

Good arguments. I will get a smaller card, and install on that with some additional space for addons. Perhaps caching is overkill. I use streams too, so seems like a waste of energy.

I will probably do sysloggin anyhow, to be able to correlate with the output from my IDP/IDS and WLAN's. That should give me be both an easy and a manageable home network.

[tsmalmbe]

If I run two identical 1GB CF/Cards, can RAID the thing then?

[Davesworld]

With a lot of work it may be possible but probably not worth the effort since we are talking flash installs and this complicates things a lot in the 1.4 branch. With the way that flash installs are set up for IPCop in general, the CF is not used much and writes are just for backup mainly so I'm not sure where there would be any gain. The Cobalt rom itself is certainly capable of booting to raid md devices though.

[tsmalmbe]

i'm looking for fault-tolerancy options. Seems like a cumbersome option to shutdown, open case, remove cf, copy, reinsert cf, close case and finally restart the firewall.

if i have two cf's (similar) in the raptor, are there tools to backup the cf card say once a day, nightly?

[Davesworld]

If there are, they are IPCop addons in general and not specific to Raqcop. In fact we all use IPCop addons on our Raqcop boxes and we do not make or supply any at all, we use the IPCop Community to find good addons like anyone else running IPcop.

This is the very reason I did not go off on a tangent with this project and make it so it would not work with IPCop addons. I stick as closely as possible to the base IPCop yet make it Cobalt friendly and useful at the same time. The whole idea of Raqcop is IPCop that will run on Cobalt hardware and give some useful lcd output. There are specific requirements to boot a Cobalt as far as the kernel goes and being headless, there are no VT's to use so they need to be commented out of inittab and rc.sysinit as well as adding ttyS0 to securetty. The LCD driver also came from the French IPCop community even though it needed some work to fit 16 char rows rather than 20 for some additional functions other than the bandwidth meter. Smartmlp one of the forum members here did some nice work adding an lcd write function that closely resembles the cobalt /sbin/lcd-write utility rather than lcd -text as we had relied on. In addition to his changes, I also removed some bloat that has been with the perl lcd program all along, specifically the ixui and dsl functions.

These changes are already in my 1.4.23cvs build tree.