CLIte/readme-internals.txt

CLIte Internals:

CLite's aim is not just to create a functional unix-like terminal in a
 web page, but also to have that built on top of a unix-like operating
 system written in javascript and which runs in a web page.

CLIte follows the common programming methodology of creating in three
 steps:

 First, make it work.
 Second, make it work right.
 Third, make it work well.

This means that some areas are still very much a work in progress, but
 are gradually being improved.

At its core, CLIte is made up of the follow parts:

The BIOS:
 The basic input and output system of CLIte, the bios provides a simplified
  api for reading input (keyboard, mouse, etc) and writing output (
  displaying text, playing sound) as well as uploading and downloading
  files to and from the CLIte system. It provides a basic interface for
  reading in keyboard input, as well as setting up the browser to provide
  a VGA textmode style graphics display, ensuring CLIte and its programs
  need never directly interact with HTML or raw events.

The Core:
 Effectively CLIte's 'kernel', this consists of functions needed to do
  basic tasks, such as loading files, loading scripts, safely running
  code, downloading files, rebooting the system, and system initialisation.
 Most of this is either abstracted away to higher level functions, or
  otherwise need never be used by user-level programs.

The Console:
 The console provides an interface between the BIOS graphics system, and
  CLIte's TTY subystem. Handling input and output to and from the currently
  active TTY. The console is located at /dev/console

The TTY:
 The tty provides a Posix-compatible terminal interface, and is linked to
  the process manager to maintain the controlling tty for each process
  group.
 Each process can interact directly with /dev/tty as a shortcut for its
  own controlling tty, which can also be found at /dev/ttyN, where N is
  the internal tty id.

The VFS: (Virtual File Sytem)
 CLIte's virtual file system bares no relation to the file system of the
  server it is run on (thus /etc/passwd is not the server's /etc/passwd
  file!).
 During system initialisation, the root filesystem is mounted by creating
  the VFS, with physical server-side files then mapped into the VFS using
  a plain-text config file which is stored in /dev/wfs (web file system,
  for lack of a better name). The content of files are then loaded in as
  needed when the VFS file is accessed.
 However other files in the VFS are created dynamically as needed during
  system initialisation. For instance, commands are loaded in directly
  from javascript functions, which creates both the executable file in
  /bin/<name> as well as the source file in /usr/src/<name>.js. Thus two
  VFS files are created from each command, with many commands being held
  in a single server-side file. Libraries are loaded in similarly, to
  both /lib/<name>.so and /usr/src/libs/<name>.js
 Additionally, configuration files in /etc, and various default devices
  in /dev are also created programatically. As is the system log file
  in /var/logs.
 Most VFS functions are abstracted away to stdio functions.

The Process Manager:
 CLIte's process manager keeps track of running programs as processes and
  process groups, in a typical unix-like manner. It also wraps each
  process in its own try/catch block to ensure errors are handled
  correctly, and that failed processes aren't left 'hanging'.
 It also manages callbacks for functions such as wait() so that processes
  can monitor and act upon each other as needed.

The User Manager:
 The user manager controls the user logins, and dynamically generates a
  guest session as needed.

The Logger:
 The Logger manages the system logs, printing them to the terminal during
  system initialisation and shutdown, as well as writing them to the
  system log at /var/logs


The Boot Process:
 1. Once the page has loaded, bios.init() is called.
	bios.video is initialised, setting up the window to work as an
	 80x25 text console.
	bios.input is initialised, setting up hidden elements for capturing
	 keyboard input, as well as for uploading files.
 2. The bios exposes a function to the global context, then loads in a
	bootable file as recorded in bios.data.bootable.
 3. core.js is loaded in, and passes clite.init to the function exposed by
	the bios.
 4. The bios calls clite.init(bios) passing its own object as an argument.
 5. CLIte begins initialising subsystems:
 6. The console is initialised, giving CLIte access to video and keyboard.
 7. The VFS is initialised, bringing up the filesystem and standard io
	 calls. The core filesystem directories are created:
		/
		/bin
		/dev
		/etc
		/lib
		/proc
		/tmp
		/usr
		/usr/clite
		/usr/clite/web
		/usr/home
		/usr/share
		/usr/share/docs
		/usr/share/site
		/usr/src
		/usr/src/libs
		/var
	As well as the system log file:
		/var/logs
 8. The TTY is initialised, extending the console to a multibuffered
	system with escape code handling. /dev/tty0 is created.
 9. The system logger changes state to write to the tty and the log file.
	Previous log data is written to the log file.
10. The root filesystem is 'mounted' by loading data/filesys.txt and
	storing it in /dev/wfs, this is then read in and server-side
	 files are mapped to the filesystem, and marked as unloaded
	 remote data.
11. The default config files are created in /etc:
	/etc/env	environment variables
	/etc/passwd	user accounts
	/etc/group	system groups
	/etc/greeting	that big CLIte text that's printed on shell login
	/etc/shrc	init file for the shell (prints the above and so on)
12. The default device files are initialised in /dev:
	/dev/local	provides an interface for upload from and
		downloading to the user's computer.
	/dev/null	the null device
	/dev/random	read from this for a random number
	/dev/initctl	for accessing and changing the system runlevel
	/dev/console	mapped to the system console
	/dev/tty	special device that is always a process'
		controlling tty
	/dev/time	for reading time in milliseconds since epoch
13. Commands and programs are loaded into /bin:
	Exposes an api for programs to load into with.
	Reads in files from the array in clite.includes.prog:
		13a: loads the file into the browser.
		13b: file content is stored in clite.commands.data as a
			function, which is then called.
		13c: each program in the file calls clite.commands.load
			passing its name and executable code as arguments.
			The executable code is stored in /bin/<name> and
			 given executable permission, the code is also
			 converted to source with .toString() and stored
			 in /usr/src/<name>.js
	The program loading code is deleted.
14. Libraries are loaded into /lib:
	Exposes an api for libraries to load into with.
	Reads in files from the array in clite.includes.libs:
		14a: loads the file into the browser.
		14b: file content is stored in clite.libs.data as a
			function, which is then called.
		14c: each library in the file calls clite.libs.load
			passing its file name, header name, and executable
			 code as arguments.
			The executable code is stored in /lib/<name>.so
			The code is also converted to source with
			 .toString() and stored in /usr/src/libs/<name>.js
			The file and header names are stored in
			 clite.libs.index for use later by the dynamic
			 linker.
15. The Process Manager is initialised, allowing for programs to be run.
16. The system is switched to runlevel 3, ending kernel initialisation,
	and ready for user-level programs to be run.
17. The guest user account's home directory is created at /usr/home/guest,
	shell dot files added, and /usr/share/site is symlinked to
	 /usr/home/guest/web
18. PID 1 is forked. At present PID 1 is /bin/login which either logs in
	automatically to the guest account, or presents a login prompt
	if another count, with a password, exists.
19. Upon succesful log in, /bin login forks a new process, sets its uid
	and gid to the new user according to /etc/passwd, then executes
	/bin/sh. At which point the shell starts up, runs its rc files,
	and presents a command prompt to the user.


The Lifecycle of a Process:

 1. fork() is called with the environment and io data passed to it.
	This adds a new process to the process manager, which assigns it
	 a process id (pid) and also creates a file in /proc/<pid> for
	 storing data about the process.
	Both the environment and io data are then cloned, and the new
	 process is called asynchronously, forking it from the current
	 process.
	The process id of the new process is returned to the parent.
 2. exec() is called, again the environment and io data is passed to it,
	along with the file path and arguments for a new program.
	This checks the file both exists and is executable by the current
	 user.
	Then updated details of the program are sent to the process manager.
	The current program is then overwritten with a call to the new
	 program.
 3. The program then runs as intended, and exits normally, or fails and
	is caught by a try/catch which exits the program.
 4. exit() is called with an integer 'exit state':
	zero for no error.
	less than zero for internal error
	greater than zero for the program's own use
	This ends the program, and notifies the process manager that the
	 process has exited.
	The process manager will then remove the process from the process
	 list, delete the process' data file in /proc/<pid>, then handle
	 any wait() calls that have been queued for the process exit.

 Typically, this is when the shell's waitpid() on the process causes the
  shell to once again show a prompt, awaiting the user's next command
  input.