Install Debian testing or Wheezy (7.0) into Acer S7

Install Debian testing or Wheezy (7.0) into Acer S7.  Dual boot with Windows 8

The steps is not what I have done exactly.  What I have done is more complicated with similar solution.  Use it at your own risk.

1. In windows, delete data partitions and shrink windows partitions
2. Download Debian testing or Wheezy (7.0) netinst iso, dd it to the USB key
3. Goto BIOS (press F2 during boot), disable secure boot, set boot from USB
4. Boot with Debian install USB key, begin install
5. Before partition, go to console 2 (Alt + F2), activate rescue console
6. Run following command to backup

# dd if=/dev/sda bs=1M | gzip -c > <a usb hard disk>/acer-s7-sda.gz
# dd if=/dev/sdb bs=1M | gzip -c > <a usb hard disk>/acer-s7-sdb.gz

They are not too big, about 10G for each.
7. After backup, continue to install
7.1 When create partition, choose "Manual", create partitions on RAID disk, NOT sda and sdb.
7.2 grub install may failed, that is not a big issue. Just continue without bootloader until install finished.
8. Boot with Debian install USB key, get the rescue console
9. chroot into the partition you installed Debian into
10. in chroot shell, do

# mount -t proc none /proc
# mount -t sysfs sysfs /sys

11. install dmraid, kpartx

# apt-get install dmraid kpartx

12. Add i915 into initramfs, this will give you a opportunity to debug during boot

# echo i915 >> /usr/share/initramfs-tools/modules

13. Add kpartx into initramfs, becuase dmraid can not recoganize gpt partition table.  Download kpartx-initramfs-tools.tar.gz, extract it, then

# cp -r kpartx-initramfs-tools/* /usr/share/initramfs-tools

14. update-initramfs

# update-initramfs -u

15. install grub-efi

# apt-get install grub-efi-amd64

16. update grub

# update-grub

17. mount efi partition?

# mkdir /boot/efi
# mount /dev/disk/by-id/dm-uuid-part2-DMRAID-isw_xxx_HDD0 /boot/efi

18. install grub

# grub-install

18. All done, reboot, and bless

X application startup script with wnck

It is common that several X applications are involved for one task. For example, web browser is used for searching and help document, Emacs is used to edit, and X termainal is used for testing. Here, script can be used to startup several X programs with one command typing or click.

But simple shell script can only startup applications. You must arrange the screen layout by hand. In fact the screen layout can be program with script too. Python + wnck can be used for that.

The script as follow is an example. It startup pidgin and gnome-terminal, minimize pidgin and maximize gnome-terminal vertically. With wnck, you can arrage the layout of your X applications arbitrarily.

#!/usr/bin/python

import gobject
import gtk
import wnck
import os
import time

def gtk_wait():
    gobject.idle_add(gtk.main_quit)
    gtk.main()

def get_wnck_apps_from_wins(wins):
    apps = set()
    for win in wins:
        app = win.get_application()
        apps.add(app)
    return list(apps)

def screen_get_wnck_apps(s):
    wins = s.get_windows()
    return get_wnck_apps_from_wins(wins)

def screen_find_windows_by_pid(s, pid):
    wins = []
    all_wins = s.get_windows()
    for win in all_wins:
        app = win.get_application()
        if app.get_pid() == pid:
            wins.append(win)
    return wins

def gtk_wait_timeout():
    gtk_wait()
    time.sleep(0.1)

def apply_timeout(func, args, timeout = 10, wait = gtk_wait_timeout):
    start = time.time()
    res = None
    while not res:
        res = apply(func, args)
        now = time.time()
        if now - start > timeout:
            break
        wait()
    return res

def wait_child_proc():
    try:
        os.waitpid(-1, os.WNOHANG)
    except OSError, err:
        if err.errno == 10: # No child processes
            pass

def pid_is_valid(pid):
    return os.path.exists('/proc/%d' % (pid,))

def screen_find_windows_by_pid_timeout(s, pid):
    while True:
        wins = apply_timeout(screen_find_windows_by_pid, (s, pid), 0.5)
        if wins:
            break
        wait_child_proc()
        if not pid_is_valid(pid):
            break
    return wins

def init():
    global screen
    screen = wnck.screen_get_default()
    gtk_wait()

class app(object):
    def __init__(self, pid):
        self.pid = pid
        self.update()
    def update(self):
        self.wnck_wins = screen_find_windows_by_pid_timeout(screen, self.pid)
        self.wnck_apps = get_wnck_apps_from_wins(self.wnck_wins)
    def minimize(self):
        for win in self.wnck_wins:
            win.minimize()
    def maximize(self):
        for win in self.wnck_wins:
            wt = win.get_window_type()
            if wt == wnck.WINDOW_NORMAL:
                win.maximize()
    def maximize_vertically(self):
        for win in self.wnck_wins:
            wt = win.get_window_type()
            if wt == wnck.WINDOW_NORMAL:
                win.maximize_vertically()
    def close(self):
        for win in self.wnck_wins:
            win.close(0)

class app_info(object):
    def __init__(self, start_argv, app_name):
        object.__init__(self)
        self.start_argv = start_argv
        self.app_name = app_name
    def get_instances(self):
        pids = []
        wnck_apps = screen_get_wnck_apps(screen)
        for wnck_app in wnck_apps:
            if wnck_app.get_name() == self.app_name:
                pids.append(wnck_app.get_pid())
        if pids:
            return [app(pid) for pid in pids]
        else:
            return [self.new_instance()]
    def new_instance(self):
        argv = self.start_argv[:]
        argv.insert(0, 'nohup')
        pid = os.spawnvp(os.P_NOWAIT, 'nohup', argv)
        return app(pid)

def start_apps():
    pidgin_info = app_info(['pidgin'], 'Pidgin')
    terminal_info = app_info(['gnome-terminal'], 'Terminal')

    os.system('rm -f nohup.out ~/nohup.out')

    apps = pidgin_info.get_instances()
    apps = pidgin_info.get_instances()
    for app in apps:
        app.close()

    apps = terminal_info.get_instances()
    for app in apps:
        app.maximize_vertically()

init()
start_apps()

Disk reading performance with hdparm

hdparm can be used to get disk reading performance with following command line:

• hdparm -T <dev>. Get cached disk reading performance. In fact, the page cache performance is tested.
• hdparm -t <dev>. Get raw disk reading performance.

Implementation

hdparm -T

ioctl(devfd, BLKFLSBUF, NULL);
read(devfd, buf, READING_SIZE)
do {
 seek(devfd, 0, SEEK_SET);
 read(devfd, buf, READING_SIZE);
} while (/* time is not over */);

Because the data to be read is in page cache always (if READING_SIZE << memory size), the result is page cache performance.

hdparm -t

ioctl(devfd, BLKFLSBUF, NULL);
do {
 read(devfd, buf, READING_SIZE);
} while (/* time is not over */);

Because the data to be read is not in page cache, and the bottleneck lies in raw disk reading speed, the result is raw disk reading performance.

reading prepare *         *         *
memory copy          -         -
raw disk read  ++++++++  ++++++++

As in above figure, reading reparation and memory copying in testing is neglectable, especially when readahead is considered.

Reading notes: Linux kernel perf event source

For Linux kernel 2.6.33

perf event enumeration

Kernel has no hardware perf event enumeration interface.

• tools command line: perf list
• Available abstract hardware perf events are hard-coded in perf user space tool. Raw hardware perf events are not enumerated by the tool, they can be specified via a 64-bit number in the tool command line. Abstract hardware perf events are implemented with the raw hardware perf events. Whether the abstract hardware perf events are available can be queried via syscall perf_event_open.
• Software trace point are exported in debugfs (/debugfs/tracing/events)

perf event management

• There is at most one perf event context for each task and each CPU. This is used to manage all perf events inside one context (task or CPU).
• All perf events in a context are linked as list in context->event_list.
• Perf events in one context are organized into groups. All group leaders in a context is linked as list in context->group_list. All perf events in a group is linked as list in group_leader->sibling_list.
• It seems that perf events in the group share some attributes, such as enable/disable, cpu, inherit, etc. In "perf record", for task events, one group is created for perf events for one CPU. In __perf_event_sched_in, it is assumed perf events in one group is for one CPU. But in "perf_record", group support can not be turned on now, that is, group is not used at all now.
• perf events inherited in child tasks are linked as list in parent_perf_event->child_list.
• In "perf record", for task profile, for one perf event type, one perf event is created for each task and each CPU. For perf event operation, it appears that one perf event for each task is sufficient, because one task can only run on one CPU at any time. But perf events may be inherited by children perf events in children tasks (forked tasks). The children perf events use the original perf events for sample output. To make sample ring buffer a per-CPU data structure, perf events are created for each CPU too. So that, when parent and child tasks run on different CPU simultaneously, they all use perf events in parent task for output, but they use different perf events for corresponding CPU.

perf event state track

• task schedule in/out (perf_event_task_sched_in/out). Record/disable counter before schedule out, restore/enable counter after schedule in. The schedule in/out event is recorded too.
• task fork
  • perf_event_fork: record fork event.
  • perf_event_init_task: perf_event->attr.inherit control whether to inherit. If inheriting, child perf event will be created for child task. Children events information is accumulated in some statistics. The sample collected via children events are sent to corresponding parent events for same CPU.
• task exit
  • perf_event_exit_task: feed back event values to parent events (child_total_time_running, etc). Free all resources for children events.
• fd closed, perf_release: free all resources. perf event in original task is managed by corresponding fd. Children perf events in children tasks are freed in perf_event_exit_task.

Dataflow

For perf event samples:

hardware counter ---> perf event mmap data page -> user space tool (perf)
                  |
soft trace point -/

For ftrace and tracing point:

 software trace point -> tracing ring buffer -> user space 

User space interface

• File based interface. File descriptor is obtained via a new syscall perf_event_open. Because one file describptor is needed for each perf event. Perf event may for a specific task, cpu, etc.
• memory map. Sample data is passed from kernel to user space mainly via memory map. Used to implemend shared memory based ring buffer. Details are in ring buffer.
• ring buffer
  • Work in both per-CPU and not-per-CPU mode
  • write side in kernel, read side in user space. shared memory communication between kernel and user space. It has better performance than memory copy.
  • lock-less write side, similar as Steven Rostedt's unified tracing buffer, perf_mmap_data.head is used to reserve space, while perf_mmap_data.user_page->data_head is used for committed data.

Code generation for software tracing point

• A set of macro is defined for software tracing point related code generation, such as binary format description, trace point function generation, etc.
• Some macros are un-defined and re-defined again and again, and some files are included again and again for different set of macro definitions. This is tricky but really powerful. Source code is in include/trace/define_trace.h and include/trace/ftrace.h.

Tracing and perf event

• Lock-less trace ring buffer is not used in perf event, another simple ring buffer implementation for mmap data pages is used instead.
• For software tracing point, the tracing point sample collecting code is shared between tracing and perf event.

Random thought

• ring buffer in perf event has some special features.
  • work in both per-CPU and not-per-CPU mode
  • shared memory communication between kernel and user space, with write side in kernel and read side in user space.
• In the code generation for software tracing point
  • macro is un-defined and re-defined again and again, and some files are included again and again for different set of macro definitions.

SWRAP: Session WRAPper

Why we need this?

Even when working on one project, usually multiple software are needed. For example, in programming, emacs is used to edit source code, one bash is used for testing, another is used for compiling. It is tedious to startup them, open latest file, type latest command etc. screen and session management of the software can be used to facilitate this. Multiple software can be started by screen automatically if you specify them in ~/.screenrc, Latest opened files can be saved by emacs desktop mechanism, latest typed command can be remember by bash history.

But it is common to work on multiple projects simultaneously. Even if you work on just one project, you may use your computer both for work and leisure. The configuration and saved state for different projects may conflict. For example, two project may use different coding style, so that different emacs configuration are needed. The bash history from one project may flush that from another project. swrap provides a way to manage mutiple configuration and saved state for each software using a simple directory based scheme. A group of software for one project can be managed with a directory hierarchy. While multiple projects are managed with multiple directory hierarchy.

Introduction

swrap stands for Session WRAPper, it provides a directory based session management for some software. The session here incude configuration files, saved state files etc. All configuration files, saved state files and other related files for each session are put into one directory. So that you can have multiple configuration, saved state at the same time. For example, you can have multiple bash histories, one for each session. This way, if you work on multiple projects simultaneously, bash history uses in one project will not flush the bash history used in another project.

Usage

swrap <command> <session directory> [<arg1> <arg2> ...]

Where

• <command> is software that is wrapped.
• <session directory> is the directory to store session related file, which can be an absolute, relative path name or a directory under ~/sessions. The base name of the <session directory> will be used as the session name at least for screen.
• <arg1> <arg2> ... are arguments for <command>.

Supported software

Bash

Configuration

• .bashrc in session directory instead of the global one will be used if provided.

Saved state

• A dedicated .bash_history in session directory will be used instead of the global one.

Emacs

Configuration

• emacs_init.el in session directory will be used as initialization file in addition to the default one.

Saved state

• Emacs desktop is used to save state, and saved state (.emacs.desktop) will be put into session directory instead of emacs startup directory or home directory.

Screen

Configuration

• screenrc in session directory will be used as initialization file instead of default one, but the default one can be loaded in this file if desired.
• screen is used to manage other software, so screen session can be seen as a compound session, while software run inside screen can be seen as sub-sesssion. So it is natural to make session directories for software run inside screen the sub-directories of screen session directory. These sub-directories can be specified as session directories for software run in screen in screenrc.

Saved state

• No saved state is supported

Example

Directory "example" shows a project session directory hierarchy managed by swrap. There are 3 software in project: screen, emacs and bash. "example" is session directory for screen, "example/screenrc" is screen configuration file. "example/0" is session directory for emacs, which is in window 0 of screen. "example/1" is session directory for bash, which is in window 1 of screen. "example/screenrc" is as follow:

source ~/.screenrc

chdir ~/projects/mce/apei/firmwarekit/linuxfirmwarekit
screen -t build 1
stuff "exec swrap bash $SESSION_DIR/1""^M"
stuff "clear""^M"

screen -t emacs 0
stuff "swrap emacs $SESSION_DIR/0 --no-window-system""^M" 

Where swrap is used to execute software in screen windows, and corresponding session directories are specified too.

When you execute:

  swrap screen <path to>/example 

or

  cp -r <path to>/example ~/sessions
  sscreen example 

You will return to your latest project state or a new project state will be created for you.

Download

swrap can be downloaded at here.

Reading notes: gnome desktop user guide

• Overall
  • Shift and Alt is often used with mouse to change the default behavior.
• Keyboard shortcut
  • Ctrl+Alt+D: show desktop
  • Ctrl+Alt+Tab: switch focus between desktop and panels
  • Alt+F9: Minimize window
• In Open/Save File Dialog
  • Show Hidden Files can be toggle with right click
  • Bookmarks can be added/removed
• When moving a panel object, alt and shift key can be used to change movement mode (switched, free, push).
• In Nautilus spatial mode, the folder can be opened with parent folder closed with "shift" pressed. I don't like leaving too much folder windows open or closing them by hand when navigating in file hierarchy.
• In Nautilus sptial mode, the browser mode can be entered with right click menu: "Browser Folder" on folder.
• Search can be saved with "File -> Save Search" in Nautilus.
• In Nautilus icon view, items can be sorted with "View -> Arrange Items -> ...".
• In Nautilus, "glob" can be used to select items with "Edit -> Select Items Matching ...".
• $HOME/Templates can be used to place arbitrary document templates, which can be used in "Create Document" menu in Nautilus.
• Nautilus sripts are placed in "$HOME/.gnome2/nautilus-scripts". Some special environment variables are used to passes some infomation.
• Custom keyboard shortcut can be added in "Keyboard Shortcuts" preference dialog directly, need not gconf-editor.

Revise muse-index

Muse is a document writing/publishing tool for Emacs.

Muse-index can be used to show all document in a muse project. But now the file name is shown and sorted by it. I think it will be more useful if document title and create or modify time is shown and sorted by create or modify time. So I revise the muse to do that. The code can be found here.