A:
Due to licensing issues we will not be releasing an alpha version of SPIN.

The code for most of the projects done using SPIN is included in the release.

Up to now, we have installed everything and ready to boot it, which I thought is not as easy as just install and compile it.

I am wondering what kind of message we should get for the "build world". I have encountered several warning message and once it stopped saying one file is permission denied so I changed the permission explicitly and the last sentence I got is like "make[1]: leaving ../user directory". I don't know whether it's ready to be booted or not, would you please tell me is there something like "compiled completely, ready to boot"?

A:
Unfortunately there are quite a few warning messages generated during the build of SPIN. As long as you don't see errors your build should be successful. The message you mention here is normal for a completed build of SPIN.

Also I think the guide you provided for installing and setting up are very helpful, but the boot part is not that clear. If you could give me more detailed info on boot part, it will be great, Would you please also tell me why SPIN need to be remote booted and what's DOS here used for?

A:
We use rconsole to remotely boot the SPIN kernel. The only other way I have booted SPIN is by booting up the crashbox under FreeBSD and ftping the kernel over from the development(Linux) machine. Then reboot the crashbox using the SPIN kernel. This is alot more time consuming than the remote boot process.

I'm not sure what information you need for booting. Could you tell me exactly what you have done and what part of the guide is confusing?

The DOS partition on the crashbox is merely used for running the salboot.com program. salboot.com is a network kernel loader program that is a DOS program. It uses bootp to discover its IP address and the server that will serve the kernel file. All of this is for remote booting.

We have tried "bootptest" on these two machines, and it works well.

Our server and crashbox are in different subnet (server:142.103.14.170; crashbox:142.103.10.32). Does this have some relation with the problem? Do you think we should make the server and crashbox in the same subnet?

Does the salboot.com broadcast in its subnet when it need to communicate with the server? Otherwise, how can it know the server's address? Thanks.

A:
BOOTP is an ethernet broadcast protocol. If your router is correctly configured, the broadcast will propogate from the client across the entire subnet to discover a server. So, yes, a bootp client and server must be on the same subnet.

Also, at the salboot.com prompt, you can set your IP addr by hand with the ip command:
>>> ip 128.95.1.1 And you can reboot salboot.com with the init command:
>>> init

Note that these commands must be run via rconsole because once salboot.com starts, all of its output goes through the serial line.

A:
I'm sorry for the problem with the download, it is an awfully large file. At this time we're not really prepared to split up the distribution. At one time I did develop the distribution as three files but the largest one was still over 15Mb as I recall.

Our bootptest is working well. And we are sure that the serial port is connected, because we can use it to copy files between the two machines.

The error message we got in rconsole is: Message Text

Can you tell us what is reason for these error messages?

Maybe we can try to boot SPIN locally. Could you send us the instruction of local boot?

Thanks a lot. We are looking forward to your reply.

A:
This error is most likely related to your network card. Unfortunately salboot.com was designed to work with a 3COM 3c595 Vortex card exclusively. You will need to obtain one of these cards for your crash machine.

Salboot Output

A:
It is not necessary to remote boot. The only other way I've booted SPIN is by ftp'ing the kernel image over to my crashbox while running FreeBSD. Then re-booting the crashbox with this kernel. Using rconsole saves you from all the ftp stuff.

How much memory does SPIN need? There is 32MB on our crashbox. Is it enough?

The error message is:
panic: pmem_alloc: out of memory
Debugger("panic")
Stopped at 0xf01226b1: movb $0,0xf02c1744
db>

A:
That means that you ran out of memory.

We have booted spin with only 32MB, but some hard-coded constants have to be changed. It will crash as is. You probably won't be able to run many services with only 32MB.

All our hosts have at least 64MB.

A:
A crash box running salboot.com will display "salboot.com" and then all further output goes to the serial line. Be sure you are connected to the serial line with rconsole before salboot.com starts up. If this is a new rconsole connection, hit a key in the rconsole window to wake up the serial line. Otherwise if you were already using the rconsole window before rebooting salboot.com, the connection will work fine.

Here is the serial output after a restart.

Can we setup SPIN on a PC with less than 64 MB of memory?

Is there a C compiler for SPIN? What other programming languages are available?

Are all the 220MB space of SPIN required? ie, are there any parts of the operating system that can be removed after setting it up? (This is because we want something small)

A:
Yes, it is possible to setup SPIN with less than 64 MB of memory. I would recommend using 64 MB however. We didn't focus on reducing the OS memory footprint when building SPIN.

You could just download C code into the SPIN kernel but the idea was to build SPIN using the type-safe properties of Modula-3.

For the most part the code under spin/kernel is required. The code under spin/user provides certain services that you may or may not want. The code under spin/local provides the Modula-3 runtime and compilers. The answer here is no, all 220 MB are not required, but what you remove will depend on what you plan on using SPIN for.

Please note that SPIN is not self-hosting. Everything you build for it will have to be cross-developed from a Linux PC.

Do you think we will need to write code into the kernel for this purpose or writing some code and compiling it normally will do?

Is SPIN stable for this purpose and does the stability depend on the stablity of Linux on which SPIN is built?
Thanks for your help

A:
The easiest way to build a firewall using SPIN is to write an extension in Modula-3 that is installed along with the other IP extensions. You can look at the tcp and udp packet handlers for examples of how to install an extension as a "packet received" event handler. The SPIN kernel uses FreeBSD 2.1.7 device drivers so some of the stability of SPIN depends on the stability of those drivers. Please note that the SPIN software is compiled/built on a Linux machine and is booted on a FreeBSD ufs partition. All of this is described in the User Guide provided with the source code.

A:
Are you referring to the http, and nfs servers? Those server processes can run on the same machine. I'm not sure exactly what your question is but I'll try to explain anyway.

You will compile/build SPIN on one machine say Machine-A. Machine-A is running Linux. Machine-A also can run the file servers nfs, http or both. You must at least run an nfs server in order to boot SPIN.

After you have built your SPIN kernel you need a second machine, Machine-B. We call this machine the crashbox or crashmachine in the User Guide. Machine-B must have a FreeBSD file partition loaded on it along with a DOS partition. This is described in the User Guide.

When you boot SPIN, the kernel is fetched from Machine-A via either http or nfs depending on how you configure this in spin/make.conf.

The message here is that you need a minimum of two machines. One is the development machine where you build SPIN and also serve the files required by SPIN(nfs and/or http). The other machine actually runs the SPIN OS.

A:
I'm not sure that I fully understand your questions. Have you compiled/built your SPIN kernel?

Once you have a fully built kernel, you will have to boot it up on a different machine from the one you built it on. Does this make sense?

How do you plan to boot SPIN? Are you planning to remotely boot using rconsole?

You need at least two machines in order to build and boot SPIN. One on which to build SPIN, the other machine is where you will actually boot and run SPIN. Forgive me if I seem redundant on this issue but I want this to be clear to you.

Are you planning on implementing the packet filter as a SPIN extension? If so then there is nothing special you need to do during setup.

Concerning setting up rconsole for SPIN, can we use the ethernet (eth0) instead of a serial connection (ttyS0)?

Why do we need this rconsole when there is another protocol used (nfs) ?

Thanks alot

A:
You want to perform a "minimal" installation of FreeBSD. This is outlined in the SPIN user guide under installing FreeBSD. The download is controlled by the installation program, you don't need to make any decisions after choosing "minimal" and selecting your installation media (ftp for installing over the net).

rconsole is designed to run over a serial connection, not ethernet.

rconsole allows you to have an interface with your SPIN machine from a remote machine. We found this to be useful for such things as restarting etc. if the SPIN machine happens to have problems.

nfs is used to transfer files between the machine where you build SPIN and the machine where you boot SPIN.

A:
I don't know the exact files that are needed for a minimum install of FreeBSD. The files are downloaded for you via ftp when you run install. I don't recall selecting a version when doing this so I imagine that the most recent version will be downloaded.

Is the problem that it takes too long to install over the network via your phone line? You may have to acquire a FreeBSD CD in order to perform the install properly.

I'm sorry I can't be of more help on this. Let me know if I misunderstand your problem and please feel free to send more questions.

Q:
We have some questions concerning pre booting procedures for the SPIN operating system:

1- When we run the rconsole from the development linux machine, should the crashbox be running salboot or freebsd?

2- Does the salboot just load and halt the machine or should it stop and do something?

3- We read in the documentation of the rconsole that we need to be running rconsole daemon on the other machine. How can we do this if it is the salboot that is running on it?

A:
The crashbox runs salboot. FreeBSD is installed only to provide a file system for SPIN to use.

When salboot starts up it merely provides a user prompt and then waits for a boot command.

The rconsole daemon doesn't run on the machine where salboot runs. In the user guide I describe a two machine scenario.

Machine_1 is the build machine. This machine is where you compile SPIN. This machine also runs these services: rconsoled, httpd, nfsd. (Note that it isn't absolutely necessary to run httpd, this is described in the user guide).

Machine_2 is your SPIN machine, the crashbox. This machine has a minimal install of FreeBSD for the file system. This machine also runs salboot.com. This is the machine where you will actually boot your SPIN kernel.

You will connect to Machine_2 from Machine_1 using rconsole. Be sure you edit the /etc/rconsoles file on Machine_1.

o Connect the two machines through their COM ports using a null modem cable.
o Start rconsoled on Machine_1.
o Run rconsole on Machine_1 using Machine_2 as the target:
    Machine_1:> rconsole Machine_2

this raises a question in my mind.... how does the ethernet interface of the crashbox know it's ip?? or does it just wait for what is comming on the serial interface??

we tried to run rconsole from our development machine while running salboot on the crashbox but it doesn't connect.... we r sure the physical link is working and the linux settings look fine... this is why we r looking for the source of the problem of the failure in connection... can u think of something that would make us sure that the salboot is intact...

when installing FreeBSD, choosing "None" in the bootloader choice does not allow us to pick between F1 to boot the dos partition or F2 to boot BSD,

however choosing the BootEasy allowed us to do that... is that ok or does it defeat the purpose of what we need...

another thing the FreeBSD makes itself the active partition after the installation is complete.. is that ok or should we make the dos partition active again...

A:
Yes salboot prints nothing on the head. salboot.com was designed for remote control so it writes its prompt to the serial line.

The IP addr salboot uses comes from bootp protocol, or by hand via salboot prompt on the serial line. I forgot to mention in the last mail that you also need to run bootp on the development machine, Machine_1 in the last mail. Once you get to the salboot prompt using rconsole, you can type help and get a list of commands. These commands allow you to set various things such as the ip addresses etc.

Does the serial line work with other OSes running on the crash box? What do you see when you type "rconsole" at at the prompt?

Be sure that you have edited your /etc/rconsoles file correctly and that the rconsoled is running.

Make the DOS partition bootable so salboot runs at startup. You can do this by running fdisk.

Q:
we have been experimenting with the serial connection for a long time at it seems that there is a problem in our serial line linux settings or the rconsole settings (although we've checked all settings quite a number of times and read all possible HOWTO's and documents)....

we'll try again later however we were wondering if u had any idea of the source of the following error (which bugged us over night)...

"rconsole: est_conn(); server host not in network database"

also would u know the number corresponding to baud 9600... we read somewhere that this number is not the one which is actually written... (for instance 9 instaed of 10NN "sorry can't remember the last 2 figures")...

A:
First of all, what does your /etc/rconsoles file look like? Can you send me the text from this file?

Do you mean the baud number in the /etc/rconsoles file? I'm afraid I don't quite understand this question.

Do you see the rconsoled process running on your machine before you try to connect using rconsole?

Q:
the crashbox is called yehia and the linux development machine is called ozz .. the rconsoles files contains one line only : ttyS1 13 ozz 2500 yehia yehia.cs.aucegypt.edu

we are working on com2 .. the connections are totally working .. we are sure about that .. cause we tried them on linux to linux .. what do you think we should investigate ..

A:
What seems to be happening is that rconsole is trying to look-up the ip address for ozz and it is failing.
Is ozz a valid host name on your network?

Can you run nslookup for ozz and get an ip address? (nslookup ozz)

Q:
after connection to the crashbox using rconsole... we tried specifying the ip of the crashbox and the server we ran the boot /home/myspin/kernel however it gives us the following error....

>>> server 208.160.131.103
server: Served by 208.160.131.103, default kernel ""
>>> ip 208.160.131.10   11
ip: 208.160.131.111.  Served by 208.160.131.103, default kernel ""
>>> b /home/myspin/kernel
mount 208.160.131.103:/home
salnet_getlocalroute: warning no if  00000000 00000000
salnet_devsend: bad args 0008DD6C 00000000 000566D4
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
salnet_devrecv: bad args 0008DD44 00000000 00000000
arp: SALNET_TIMEOUT
fetch: SALNET_TIMEOUT

we are pretty confident in our mount and nfs settings however we are in doubt of bootpd... do you share this doubt with us??

a different approach we tried was:
after trying out rconsole our best guess was that something was wrong with the bootpd settings... so we removed the simple problems except one that we couldn't trace to anything....

when typing "bootpd -d4" to get debug information from a static bootpd session we would get:

bootpd: info(6):   bootptab mtime: Tue Feb 16 21:56:37 1999
bootpd: info(6):   reading "/etc/bootptab"
bootpd: info(6):   read 3 entries (1 hosts) from "/etc/bootptab"
bootpd: error(3):  bind: Address already in use

we added tftp functionality to our linux server as it is conventionally needed by bootp however it didn't work either.... do we really need tftp??

one last question... we don't have NIS on our server.. do we need to run NIS especially for the physical ether addresses or is the bootptab enough??

A:
This error may be related to your network card. What type of card do you have in your crashbox? We only modified a couple of drivers for SPIN and I suspect that salboot.com only works with the 3COM 3c905-tx (Vortex) card.

When you boot up the crashbox salboot.com should print out some information about your ethercard before displaying the >>> prompt. If you don't see this then salboot may not be recognizing your card.

Q:
we have one concern related to the hardware .. after you told us that its the 3com905 ethernet card that is needed not another brand .. we are now worried about the display card .. does it have to be Matrox Millenium .. or can it be any standard display card ..
thanks ..

another question .. that worried us again .. is there the plexus networking architecture on pcs rather than dec alpha .. or at least is there a tcp/ip implementation on pcs .. we are concerned aout this because we are working on pcs over here and we need a tcp/ip implementation at least to continue with our project .. is the plexus or the tcp/ip implementation provided with the distribution .. thanks again ..

A:
The video card only matters for running X11 servers and we never did one for the PCs so you don't need to worry about that.

The TCP/IP implementation is hardware independent so it runs on PC and alpha.

Q:
I need some hints because I have to make a thesis on SPIN, and the Venice University is reluctant to give me to machine in order to do it.

- Does a precompiled version of SPIN's kernel exist?
- Is it mandatory to boot SPIN over a network?

A:
I don't quite understand what you would like to do. A precompiled version of the SPIN kernel does exist. In order to use it you will need a i386(PC) machine that has a FreeBSD ufs partition.

A kernel that is compiled from the source distribution is configured to dynamically fetch(retrieve) extensions over the network. If you have a Linux computer you can build your own kernel and set of extensions. It is possible to configure the kernel so that the extensions are contained within the kernel itself. This would remove the network boot dependencies. The documentation on this process is not up to date so in order to do this you would have to become familiar with the build procedure. I do not know how this would be done myself either.

I hope this answers your questions. Feel free to send more questions as the come up.

Q:
here is our situation ..
we have two ethernet cards on our crashbox because we want to use it as a router .. the two cards are detected by freebsd with no irq conflicts ... but it gives us this error ... where do you think the problem is .. in salboot or what ...

SALBOOT.COM
Probing for devices on the ISA bus:
Probing for devices on PCI bus 0:
pci0:0: Intel Corporation, device=0x00007180, class=bridge (host)
pci0:1: Intel Corporation, device=0x, class=bridge (pci)
pci0:7:0: Intel Corporation, device=0x, class=bridge (isa)
pci0:7:1: Intel Corporation, device=0x, class=storage (ide) [no driver assigned]
pci0:7:2: Intel Corporation, device=0x, class=0x0000000C,
subclass=0x00000003 int d irq 11 [no driver assigned]
pci0:7:3: Intel Corporation, device=0x, class=bridge (misc)
vx0 <3COM 3C595 Fast Etherlink XL PCI> rev 0 int a irq 9 on pci0:13
mii[*mii*] address 00:10:4b:87:44:b5
salnet_bootdev: vx0
vx1 <3COM 3C595 Fast Etherlink XL PCI> rev 0 int a irq 5 on pci0:14
vx1: not configured; kernel is built for only 1 device.
bootp: 208.160.131.103. Served by
yaaa.cs.aucegypt.edu(208.160.131.111), default kernel "/home/null"
>>> Attempting to connect to crab (via /dev/ttyS1) ... Connected
rconsole: Type '^_{q|e}' to exit.
esc "quit" (exit client)

A:
Yes, I think this error comes from salboot.com, it doesn't need more than one network network device. Did you try and boot your spin kernel anyway? It looks as though you had the rconsole prompt ">>>" but then you exited.

A:
Actually no one here has utilized two network cards on one machine in the context of SPIN.

My problem is that I could not have root access on the Linux machine in order to set up it as a boot server.

A:
It is possible to boot your crash-machine using FreeBSD, transfer your SPIN kernel(via ftp) from your build (Linux) machine over to your crash-machine (FreeBSD), then re-boot the crash-machine using the SPIN kernel. SPIN will still require network services such as nfs and http to boot.

1) after the installation of SPIN, is it possible to use it as a stand-alone machine not connected to any network?

2) In your installation instructions you use rconsole. I can't use it because the two machines are very distant. Is this a problem?

A:
Since we've never run SPIN without a network connection, it is not clear to people here whether it will boot correctly without a network connection.

Also, since your machine running SPIN doesn't have a network connection nor can be connected via rconsole I still don't see how you are going to boot SPIN. I suppose it could just boot off of disk and then implement some fixed function, but we've never done something like that.

It probably would be best if you get SPIN to work in a networked/rconsoled environment to start out with. With any other scenario we won't be able to provide help other than just conjecture.

The file log.txt attached to this email report the compilation output.

It seems that the linker could not find libc, but I had no problems compiling the modula-3 compiler and the bsd-tools.

A:
Here is what our "SPIN build expert" says:

My guess is that the static libc.a is not installed, and this host only has the dynamic libc.so. All the missing symbols are libc functions and our M3 build makes static programs. I've seen linux boxes before that never had the static libc installed so this is my first thought.

one of the extensions that we have written imports FileWr and FileRd. Both those two files are assumed to be provided be the fs extension, (the documentation is found at /spin/libm3_sa/src/rw/doc/rw.html)

However, our extension can't import those files. We have tried to include these files in the /spin/libm3_sa/src/rw/ directory along with the files it imports. We then recompiles the libm3_sa library. Every thing worked fine, bet we had a linking error. We also tried to add the FileWr to the fs project under the directory /spin/user/fs/fscore/src/ and we tried to recompile the fs, but it didn't work for some reason.

Actually we tried to find any object files in the spin tree for FileWr (ie, FilwWr.io or FilwWr.mo) but there is no such files.

A:
I have a few suggestions that I hope will be helpful.

There is a thing called extent file system (efs). It's in user/fs/efs. Efs allows you to create files with fixed sizes and delete them. Take a look at spin/user/fs/efs/doc/index.html.

No one here has done much work that involves creating, reading, and writing files so you are exploring new areas in your work. The easiest way to create a file is through a script that executes shell commands. Here is how you can create a file on efs.

nanny touch Extent
nanny touch efs
extent mkdev $EDISKDEV efsextent -size 167772160
mkdir /efs
mount efs efsextent /efs
mkfile /efs/foo 23886080

rm /efs/foo

nanny touch Extent
mkdir /efs
mount efs $EDISKDEV /efs
mkfile /efs/foo 23886080

NOTE: These series of shell commands resulted in an error that hasn't been tracked down. A work-around is to insert an extra command prior to the mkfile command that will add the filename to the nameserver.

touch /efs/foo

1) When salboot.com starts, the system seems blocked: nothing appears on the screen. I guess that salboot is waiting for a connection via the serial line. Is this behavior normal?

2) After FreeBSD Release 3.1 installation, if I try to boot FreeBSD, the boot process prints something about hostname setup; at this point the system is blocked. Is it normal?

A:
Yes salboot prints nothing on the head. salboot.com was designed for remote control so it writes its prompt to the serial line.

This I am not familiar with. I don't believe this is normal. Perhaps your installation of FreeBSD did not complete correctly? What is the exact message that you see at boot time?

A:
As far as unloading the extensions are concerned, there isn't a good mechanism designed for doing this. The problem was that we never designed a mechanism to keep track of the extension dependencies that result from loading various extensions on top of each other. To remove one might leave others dependent upon code that no longer existed.

In practice, most extensions had a shell command as part of the extension. We often added a 'zap' argument that would delete the command from the shell, clean up any dispatcher handlers that were installed, and anything else needed to make way for a re-installation. After the 'zap', we installed the next version of the extension in the typical manner. So the memory with the first version of the extension is still there, but there are no more links to it so it does not interfere with the new version of the extension.

See "user/shell/basiccmds/src/Show.m3" for an example of zap.

I looked at this myself and was confused because of the call to ShowCmd.Uninstall(). I couldn't find ShowCmd.m3. ShowCmd.*3 are autogenerated files under user/shell/basiccmds/IX86_SPIN/, you won't see them because Show.m3 is not built by default, see user/shell/basiccmds/src/m3makefile.

Anyway, you can add a 'zap' command to your extension and that will allow you to reload it. Follow the example in Show.m3 and be sure to list your extension in the m3makefile like this:
    CommandModule("extensionName")
This should cause the extensionNameCmd.*3 files to be generated according to the template.

Also be aware that the Uninstall method removes the extensionName.Run handler from the Shell.Run event. This is all for just the shell command itself. Your extension proper will have to remove whatever other events handlers it installs.

What if we just used the Extent file system ( /spin/user/fs/efs ). Actually I tried using the interfaces of this extension, but it keeps giving me a linking error ...

I feel that some thing is missing in the file system implementation, but I can't find out what it is ...

A:
Yes, I recommend either using the extent file system or you could try mounting a file system from a different machine on your network.

what we need to do is to split the packet into a header and data part in order to send it using the IpGen.PacketSend() function... we do not need to change anything in the packet... however this doesn't seem to work...

we are guessing that the problem is that we mess up the packet data while splitting it... probably the data part... this is because the interfaces for Mbuf are quite obscure and we can't really understand them...

if u have a simpler way to just split the packet into two parts to directly use Packet Send we would greatly appreciate it.... or at least some kind of guide on how to use the Mbuf interface...

by the way the firewall does work on spin however if we can't fix this problem we will get the packets... filter them but not send them across to the other network... which would be a great problem in our project....

one last question... which is not really crucial for our problem but related... does the icmpClient allow pings across networks... cause we think (by reading the code and trying it) that whenever an icmp packet arrives.. it flips the ip in the packet and resends it.... this has been the source of confidence that the IpGen.PacketSend() really works... cause it calls it at the end of the IcmpGen.PacketSend() function... so we would want to know if what we have just mentioned is true....

this is the code fragemant that supposidly copies the data part from the original Mbuf.T into this.pkt which is also Mbuf.T

Code Fragment

A:
What is the failure that you see? Can you be more specific about what is going wrong?

Are you successful in sending a packet, but that packet is corrupt?

another problem that we have is that the total length of the packet using the ip header tot_len is claimed to be 11281 (aprox can't remember exact one) however the Mbuf using m_length() is 46.... which one is more truthful???

A:
This seems like a problem. I'd think that the 46 is the truthful one. ALthough we got SPIN to work on x86, our main platform was the alpha. I wouldn't be surprised if the tot_len value is not initialized correctly. Additionally, the tot_len value can only be used from an mbuf that is a special hdr mbuf. I.e., one that has the 'hdr' bit set in the mbuf m_flags field. If that bit is not set, then the tot_len actually contains some data. I realize that the mbuf stuff is pretty hard, and in retrospect that was a mistake in our development of the protocol stack.

Another reason why your linux box might not accept the packet is if the tcp checksum values aren't right. I am assuming that your firewall code recalculates the tcp checksum value correctly when you change the IP address in order to redirect the packet.

if u can just tell us how to send this packet again we would be very grateful...

A:
I just realized that you say you want to "resend" an MBuf.T. Does this mean that you are trying to send the same MBuf.T more than once? If so then I think this will not work.

The networking code will implicitly free the MBuf.T once the packet is sent. You should not keep a pointer to it. Instead, you should keep a pointer to the data embodied in the MBuf. That way, you can always create a new MBuf wrapped around the same chunk of data, and thus resend an identical packet. You can check out the web server to see how the web server caches data pages corresponding to web pages, and reuses them by wrapping new mbufs around them. This is in user/httpd/src as I mentioned to you before.