setup_arch(3) - local_async_enable()

看完boot arguments之後繼續往下看setup_arch() 的code

[local_async_enable]

         /*
          *  Unmask asynchronous aborts after bringing up possible earlycon.
          * (Report possible System Errors once we can report this occurred)
          */
         local_async_enable();

local_async_enable 的code 在 arch/arm64/include/asm/irqflags.h#L59

 #define local_async_enable()    asm("msr        daifclr, #4" : : : "memory")
 #define local_async_disable()   asm("msr        daifset, #4" : : : "memory")

這邊是將 SPSR中的的A bit設定起來

對於SPSR 等system register只能透過msr or mrs 來存取

而 這些register中某些特定的欄位可以直接指定要 set 還是 clear

• MSR DAIFClr, #imm4 : clear any or all of DAIF to 1

• MSR DAIFSet, #imm4 : set any or all of DAIF to 0
• MSR SPSel, #imm4 : select stack pointer between EL0 to ELn

DAIF指得就是如下圖SPSR中的DAIF 四個field

According to ARM Programmers Guide for ARMv8
----------------------------------------------------------------------------------------------------
D: Process state Debug mask. Indicates whether debug exceptions from watchpoint,
breakpoint, and software step debug events that are targeted at the Exception level
the exception occurred in were masked or not
若mask, 代表發生exception時core會知道
----------------------------------------------------------------------------------------------------
A: SError (System Error) mask bit.
若mask, 代表發生system error時, core 會知道
----------------------------------------------------------------------------------------------------
I: IRQ mask bit.
若mask, 代表IRQ不會被core 知道(IRQ不會打進來)
----------------------------------------------------------------------------------------------------
F: FIQ mask bit.
若mask, 代表FIQ不會被core 知道(FIQ不會打進來)
----------------------------------------------------------------------------------------------------

[History]
這一篇mailing list [3]說明了為什麼要加這個function
subject
[arm64: Unmask asynchronous aborts when in kernel mode]
The asynchronous aborts are generally fatal for the kernel but they can be masked via the pstate A bit. If a system error happens while in kernel mode, it won't be visible until returning to user space. This patch enables this kind of abort early to help identifying the cause.

作者提到當系統在kernel mode發生error時, 必須要等到return 到user space才會被看見

因此為了讓 abort 提早被發現, 他在 setup.c的setup_arch() 與 smp.c 的 secondary_start_kernel()中加入這個patch, 將可以看到system error的bit enable起來

[Asynchronous\Synchronous Exception]

個別說明兩種exception的差別

An exception is described as synchronous if it is generated as a result of execution or attempted execution of the instruction stream, and where the return address provides details of the instruction that caused it.

Synchronous exception 發生的原因如下

• Instruction aborts from the MMU. For example, by reading an instruction from a memory location marked as Execute Never.
• Data Aborts from the MMU. For example, Permission failure or alignment checking.
• SP and PC alignment checking.
• Synchronous external aborts. For example, an abort when reading translation table.
• Unallocated instructions.
• Debug exceptions.

An asynchronous exception is not generated by executing instructions, while the return address might not always provide details of what caused the exception.
非同步例外: 是由 IRQ (normal priority interrupt), FIQ (fast interrupt) or SError (System Error)所引起. For example, an abort triggered by writeback of dirty data from a cache line to external memory

[reference]
1. http://www.voidcn.com/blog/longwang155069/article/p-6147749.html
2. http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0024a/CHDDJBAB.html
3. http://marc.info/?l=git-commits-head&m=138574902028026&w=2

setup_arch (2) - Kernel boot arguments_parse_early_param()

[Bootloader傳什麼進入kernel]
Bootloader 會傳入一串參數給kernel 以決定kernel 的運行
傳給核心的參數是以空格分隔的字串，通常的型式是
param[=value_1][,value_2]...[,value_10]

param是keyword, 一個param後面可以接最多10個value

由 bootloader 傳給核心的參數字串也可以包含傳給 init 程序的參數，kernel只會解析到 "--" 之前的字串，在 "--" 之後的字串會被當成傳給 init 程序的參數

[Bootloader用什麼data struct 傳入kernel]

使用struct tag, define 如下

 struct tag {
         struct tag_header hdr;
         union {
                 struct tag_core         core;
                 struct tag_mem32        mem;
                 struct tag_videotext    videotext;
                 struct tag_ramdisk      ramdisk;
                 struct tag_initrd       initrd;
                 struct tag_serialnr     serialnr;
                 struct tag_revision     revision;
                 struct tag_videolfb     videolfb;
                 struct tag_cmdline      cmdline;
 
                 /*
                  * Acorn specific
                  */
                 struct tag_acorn        acorn;
 
                 /*
                  * DC21285 specific
                  */
                 struct tag_memclk       memclk;
         } u;
 };

每一個tag都有一個tag_header定義如下

  struct tag_header {
          __u32 size;
          __u32 tag;
  };

[When will Bootloader pass into kernel]

 void __init setup_arch(char **cmdline_p)
 {
         pr_info("Boot CPU: AArch64 Processor [%08x]\n", read_cpuid_id());
 
         sprintf(init_utsname()->machine, ELF_PLATFORM);
         init_mm.start_code = (unsigned long) _text;
         init_mm.end_code   = (unsigned long) _etext;
         init_mm.end_data   = (unsigned long) _edata;
         init_mm.brk        = (unsigned long) _end;
 
         *cmdline_p = boot_command_line;
 
         early_fixmap_init();
         early_ioremap_init();
 
         setup_machine_fdt(__fdt_pointer);
 
         parse_early_param();
 ...

從start_kernel --> setup_arch --> parse_early_param
進到parse_early_param就會開始解析preloader傳進來的參數

[early_param]
有較高優先權的參數稱為early_param
以maxcpus為例, 由preloader 傳給kernel, 因為(when __SMP__ is defined)boot up 時就需要知道maxcpus是多少, 因此在early_param時就會執行 maxcpus() in kernel/smp.c
More 可以參考
http://man7.org/linux/man-pages/man7/bootparam.7.html

 static int __init maxcpus(char *str)
 {
         get_option(&str, &setup_max_cpus);
         if (setup_max_cpus == 0)
                 arch_disable_smp_support();
 
         return 0;
 }
 
 early_param("maxcpus", maxcpus);

至於early_param 這個macro又是什麼呢? 繼續往下看
include/linux/init.h

 /*
  * Only for really core code.  See moduleparam.h for the normal way.
  *
  * Force the alignment so the compiler doesn't space elements of the
  * obs_kernel_param "array" too far apart in .init.setup.
  */
 #define __setup_param(str, unique_id, fn, early)                        \
         static const char __setup_str_##unique_id[] __initconst         \
                 __aligned(1) = str;                                     \
         static struct obs_kernel_param __setup_##unique_id              \
                 __used __section(.init.setup)                           \
                 __attribute__((aligned((sizeof(long)))))                \
                 = { __setup_str_##unique_id, fn, early }
 
 #define __setup(str, fn)                                                \
         __setup_param(str, fn, fn, 0)
 
 /*
  * NOTE: fn is as per module_param, not __setup!
  * Emits warning if fn returns non-zero.
  */
 #define early_param(str, fn)                                            \
         __setup_param(str, fn, fn, 1)

early_param 跟 __setup 的實作都是 __setup_param, 只不過 early_param --> __setup_param會將其中的early參數設為1

進入__setup_param會發現這個macro其實是define兩個variable
(1) define 一個型態為 static const char 的variable, 並將str assign給它
     __setup_str_##unique_id[] = str

(2) define 一個型態為 static struct obs_kernel_param 的variable __setup_##unique_id
      並初始化data structure
      __setup_##unique_id = {__setup_str_##unique_id, fn, early}

struct obs_kernel_param 可以讓kernel記錄字串參數與相對應的處理函式
一樣定義在 include/linux/init.h

 struct obs_kernel_param {
         const char *str; //name of params
         int (*setup_func)(char *); // function handler
         int early; //true if it's early_param
 };

我們再一次用上面maxcpus來展開

 static int __init maxcpus(char *str)
 {
         get_option(&str, &setup_max_cpus);
         if (setup_max_cpus == 0)
                 arch_disable_smp_support();
 
         return 0;
 }
 
 early_param("maxcpus", maxcpus);

macro 炸開之後........

static const char __setup_str_maxcpus[] __initconst __aligned(1) = "maxcpus"

static struct obs_kernel_param __setup_maxcpus __used __section(.init.setup) __attribute__((aligned(sizeof(long)))))

= { __setup_str_maxcpus, maxcpus, 1};

這樣就可以很清楚的發現
我們定義了一個裝有"maxcpus"的字串陣列, 再來有一個記錄 kernel params的 struct obs_kernel_param, 用來告訴kernel 這個param叫什麼, 以及遇到這個param的時候開call 什麼fundtion handler來處理

另外用藍色標起來的部分又是一個難題了
這裡將這個macro定義的資料放在 .init.setup 這個section中
關於.init.setup 這個section 是定義在 kernel-4.4\include\asm-generic\vmlinux_lds.h 中

#define INIT_SETUP(initsetup_align) \
. = ALIGN(initsetup_align); \
VMLINUX_SYMBOL(__setup_start) = .; \
*(.init.setup) \
VMLINUX_SYMBOL(__setup_end) = .;

代表所有被標註要放到.init.setup這個section的params都會被放在一起
而起始為至是 __setup_start & __setup_end

怎麼知道到底有哪些params會被放在 .init_setup中呢
可以打開 out\System.map看看
找到__setup_start了!! 下面緊接著都是 __setup_XXX
還看到了前面的舉例 __setup_maxcpus 這個param

[parse_early_param]
回到start_kernel--> setup_arch--> parse_early_param
這裡將boot_command_line copy 給 tmp_cmdline
接著呼叫 parse_early_options--> parse_args 開始解析kernel parameters
parse_args 會將cmd args切成一組組的<param,value>, 傳入callback function do_early_param

 void __init parse_early_options(char *cmdline)
 {
         parse_args("early options", cmdline, NULL, 0, 0, 0, NULL,
                    do_early_param);
 }
 
 /* Arch code calls this early on, or if not, just before other parsing. */
 void __init parse_early_param(void)
 {
         static int done __initdata;
         static char tmp_cmdline[COMMAND_LINE_SIZE] __initdata;
 
         if (done)
                 return;
 
         /* All fall through to do_early_param. */
         strlcpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE);
         parse_early_options(tmp_cmdline);
         done = 1;
 }

[do_early_param]
將傳入的params 跟 __setup_start , __setup_end區間中的args做比對, 若有符合的, 就call 該args的callback function (p->setup_func)做設定

 /* Check for early params. */
 static int __init do_early_param(char *param, char *val,
                                  const char *unused, void *arg)
 {
         const struct obs_kernel_param *p;
 
         for (p = __setup_start; p < __setup_end; p++) {
                 if ((p->early && parameq(param, p->str)) ||
                     (strcmp(param, "console") == 0 &&
                      strcmp(p->str, "earlycon") == 0)
                 ) {
                         if (p->setup_func(val) != 0)
                                 pr_warn("Malformed early option '%s'\n", param);
                 }
         }
         /* We accept everything at this stage. */
         return 0;
 }

reference:
[1] https://danielmaker.github.io/blog/linux/kernel_parameter_parsing.html
[2] http://blog.csdn.net/goto_chen/article/details/17392245
[3] http://blog.csdn.net/skyflying2012/article/details/41142801

Fix-Mapped Linear Address

Architecture: ARM64
[Intro]

Fixmap是固定一直指到physical addr的特定位址

Kernel會一次配一塊4K的virtual memory mapping 到physical addr

[Usage of Fix-Mapped Linear Address]

Kernel linear address的第四個GB中至少會有一塊128MB的memory mapping到physical address

這一塊memory 可以讓kernel implement 

(1) noncontiguous memory allocation (2) fix-mapped linear address

[Fix-Mapped Linear Address VS. Physical Address]

Fix-Mapped Linear Address 是一個constant address. 例如 0xffffc000

每一個Fix-Mapped Linear Address maps到一個page frame(4K) 的physical memory

Fix-Mapped Linear Address 跟 linear address that map the first 896MB of RAM 很像

但Fix-Mapped Linear Address可以mapping到任何physical address

[Data Structure enum fixed_addresses]

每一個fix-mapped linear address都有一個專用的integer index defined 在 enum fixed_addresses中

  enum fixed_addresses {
          FIX_HOLE,
  
          /*
           * Reserve a virtual window for the FDT that is 2 MB larger than the
           * maximum supported size, and put it at the top of the fixmap region.
           * The additional space ensures that any FDT that does not exceed
           * MAX_FDT_SIZE can be mapped regardless of whether it crosses any
           * 2 MB alignment boundaries.
           *
           * Keep this at the top so it remains 2 MB aligned.
           */
  #define FIX_FDT_SIZE            (MAX_FDT_SIZE + SZ_2M)
          FIX_FDT_END,
          FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
  
          FIX_EARLYCON_MEM_BASE,
          FIX_TEXT_POKE0,
          __end_of_permanent_fixed_addresses,
  
          /*
           * Temporary boot-time mappings, used by early_ioremap(),
           * before ioremap() is functional.
           */
  #define NR_FIX_BTMAPS           (SZ_256K / PAGE_SIZE)
  #define FIX_BTMAPS_SLOTS        7
  #define TOTAL_FIX_BTMAPS        (NR_FIX_BTMAPS * FIX_BTMAPS_SLOTS)
  
          FIX_BTMAP_END = __end_of_permanent_fixed_addresses,
          FIX_BTMAP_BEGIN = FIX_BTMAP_END + TOTAL_FIX_BTMAPS - 1,
  
          /*
           * Used for kernel page table creation, so unmapped memory may be used
           * for tables.
           */
          FIX_PTE,
          FIX_PMD,
          FIX_PUD,
          FIX_PGD,
  
          __end_of_fixed_addresses
  };

[How to Obtain the Linear Address Set of a Fix-Mapped Linear Address]

 #ifndef __ASSEMBLY__
  /*
   * 'index to address' translation. If anyone tries to use the idx
   * directly without translation, we catch the bug with a NULL-deference
   * kernel oops. Illegal ranges of incoming indices are caught too.
   */
  static __always_inline unsigned long fix_to_virt(const unsigned int idx)
  {
          BUILD_BUG_ON(idx >= __end_of_fixed_addresses);
          return __fix_to_virt(idx);
  }

fix_to_virt -> __fix_to_virt: 能夠根據index找到 constant linear address
如下, PAGE_SHIFT = 12
所以每一個index 對應到的linear address是從FIXADDR_TOP開始減 4K 的倍數

 #define __fix_to_virt(x)        (FIXADDR_TOP - ((x) << PAGE_SHIFT))

目前畫出的memory layout不一定正確, 若有誤會再修正

[Reference]
Fixmap:
http://palliatory66.rssing.com/chan-60693167/latest.php

ARM64 memory
http://blog.csdn.net/qianlong4526888/article/details/9058221

Linux doc about arm64 memory
http://lxr.free-electrons.com/source/Documentation/arm64/memory.txt?v=4.4

Linux kernel memory management
https://0xax.gitbooks.io/linux-insides/content/mm/linux-mm-2.html

Windows7 memory layout
http://www.codemachine.com/article_x64kvas.html