2016年10月25日 星期二

setup_arch (2) - Kernel boot arguments_parse_early_param()

[Bootloader傳什麼進入kernel]
Bootloader 會傳入一串參數給kernel 以決定kernel 的運行
傳給核心的參數是以空格分隔的字串,通常的型式是
param[=value_1][,value_2]...[,value_10]
param是keyword, 一個param後面可以接最多10個value
由 bootloader 傳給核心的參數字串也可以包含傳給 init 程序的參數,kernel只會解析到 "--" 之前的字串,在 "--" 之後的字串會被當成傳給 init 程序的參數

[Bootloader用什麼data struct 傳入kernel]
使用struct tag, define 如下
146 struct tag {
147         struct tag_header hdr;
148         union {
149                 struct tag_core         core;
150                 struct tag_mem32        mem;
151                 struct tag_videotext    videotext;
152                 struct tag_ramdisk      ramdisk;
153                 struct tag_initrd       initrd;
154                 struct tag_serialnr     serialnr;
155                 struct tag_revision     revision;
156                 struct tag_videolfb     videolfb;
157                 struct tag_cmdline      cmdline;
158 
159                 /*
160                  * Acorn specific
161                  */
162                 struct tag_acorn        acorn;
163 
164                 /*
165                  * DC21285 specific
166                  */
167                 struct tag_memclk       memclk;
168         } u;
169 };
每一個tag都有一個tag_header定義如下
 24 struct tag_header {
 25         __u32 size;
 26         __u32 tag;
 27 };
[When will Bootloader pass into kernel]
227 void __init setup_arch(char **cmdline_p)
228 {
229         pr_info("Boot CPU: AArch64 Processor [%08x]\n", read_cpuid_id());
230 
231         sprintf(init_utsname()->machine, ELF_PLATFORM);
232         init_mm.start_code = (unsigned long) _text;
233         init_mm.end_code   = (unsigned long) _etext;
234         init_mm.end_data   = (unsigned long) _edata;
235         init_mm.brk        = (unsigned long) _end;
236 
237         *cmdline_p = boot_command_line;
238 
239         early_fixmap_init();
240         early_ioremap_init();
241 
242         setup_machine_fdt(__fdt_pointer);
243 
244         parse_early_param();
245 ...

從start_kernel --> setup_arch --> parse_early_param
進到parse_early_param就會開始解析preloader傳進來的參數

[early_param]
有較高優先權的參數稱為early_param
以maxcpus為例, 由preloader 傳給kernel, 因為(when __SMP__ is defined)boot up 時就需要知道maxcpus是多少, 因此在early_param時就會執行 maxcpus() in kernel/smp.c
More 可以參考
http://man7.org/linux/man-pages/man7/bootparam.7.html
524 static int __init maxcpus(char *str)
525 {
526         get_option(&str, &setup_max_cpus);
527         if (setup_max_cpus == 0)
528                 arch_disable_smp_support();
529 
530         return 0;
531 }
532 
533 early_param("maxcpus", maxcpus);
至於early_param 這個macro又是什麼呢? 繼續往下看
include/linux/init.h
238 /*
239  * Only for really core code.  See moduleparam.h for the normal way.
240  *
241  * Force the alignment so the compiler doesn't space elements of the
242  * obs_kernel_param "array" too far apart in .init.setup.
243  */
244 #define __setup_param(str, unique_id, fn, early)                        \
245         static const char __setup_str_##unique_id[] __initconst         \
246                 __aligned(1) = str;                                     \
247         static struct obs_kernel_param __setup_##unique_id              \
248                 __used __section(.init.setup)                           \
249                 __attribute__((aligned((sizeof(long)))))                \
250                 = { __setup_str_##unique_id, fn, early }
251 
252 #define __setup(str, fn)                                                \
253         __setup_param(str, fn, fn, 0)
254 
255 /*
256  * NOTE: fn is as per module_param, not __setup!
257  * Emits warning if fn returns non-zero.
258  */
259 #define early_param(str, fn)                                            \
260         __setup_param(str, fn, fn, 1)

early_param 跟 __setup 的實作都是 __setup_param, 只不過 early_param --> __setup_param會將其中的early參數設為1

進入__setup_param會發現這個macro其實是define兩個variable
(1) define 一個型態為 static const char 的variable, 並將str assign給它
     __setup_str_##unique_id[] = str

(2) define 一個型態為 static struct obs_kernel_param 的variable __setup_##unique_id
      並初始化data structure
      __setup_##unique_id = {__setup_str_##unique_id, fn, early}

struct obs_kernel_param 可以讓kernel記錄字串參數與相對應的處理函式
一樣定義在 include/linux/init.h
232 struct obs_kernel_param {
233         const char *str; //name of params
234         int (*setup_func)(char *); // function handler
235         int early; //true if it's early_param
236 };
我們再一次用上面maxcpus來展開
524 static int __init maxcpus(char *str)
525 {
526         get_option(&str, &setup_max_cpus);
527         if (setup_max_cpus == 0)
528                 arch_disable_smp_support();
529 
530         return 0;
531 }
532 
533 early_param("maxcpus", maxcpus);
macro 炸開之後........
static const char __setup_str_maxcpus[] __initconst __aligned(1) = "maxcpus"
static struct obs_kernel_param __setup_maxcpus __used __section(.init.setup) __attribute__((aligned(sizeof(long)))))
= { __setup_str_maxcpus, maxcpus, 1};

這樣就可以很清楚的發現
我們定義了一個裝有"maxcpus"的字串陣列, 再來有一個記錄 kernel params的 struct obs_kernel_param, 用來告訴kernel 這個param叫什麼, 以及遇到這個param的時候開call 什麼fundtion handler來處理

另外用藍色標起來的部分又是一個難題了
這裡將這個macro定義的資料放在 .init.setup 這個section中
關於.init.setup 這個section 是定義在 kernel-4.4\include\asm-generic\vmlinux_lds.h 中

#define INIT_SETUP(initsetup_align) \
. = ALIGN(initsetup_align); \
VMLINUX_SYMBOL(__setup_start) = .; \
*(.init.setup) \
VMLINUX_SYMBOL(__setup_end) = .;

代表所有被標註要放到.init.setup這個section的params都會被放在一起
而起始為至是 __setup_start & __setup_end

怎麼知道到底有哪些params會被放在 .init_setup中呢
可以打開 out\System.map看看
找到__setup_start了!! 下面緊接著都是 __setup_XXX
還看到了前面的舉例 __setup_maxcpus 這個param







[parse_early_param]
回到start_kernel--> setup_arch--> parse_early_param
這裡將boot_command_line copy 給 tmp_cmdline
接著呼叫 parse_early_options--> parse_args 開始解析kernel parameters
parse_args 會將cmd args切成一組組的<param,value>, 傳入callback function do_early_param
430 void __init parse_early_options(char *cmdline)
431 {
432         parse_args("early options", cmdline, NULL, 0, 0, 0, NULL,
433                    do_early_param);
434 }
435 
436 /* Arch code calls this early on, or if not, just before other parsing. */
437 void __init parse_early_param(void)
438 {
439         static int done __initdata;
440         static char tmp_cmdline[COMMAND_LINE_SIZE] __initdata;
441 
442         if (done)
443                 return;
444 
445         /* All fall through to do_early_param. */
446         strlcpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE);
447         parse_early_options(tmp_cmdline);
448         done = 1;
449 }

[do_early_param]
將傳入的params 跟 __setup_start , __setup_end區間中的args做比對, 若有符合的, 就call 該args的callback function (p->setup_func)做設定
411 /* Check for early params. */
412 static int __init do_early_param(char *param, char *val,
413                                  const char *unused, void *arg)
414 {
415         const struct obs_kernel_param *p;
416 
417         for (p = __setup_start; p < __setup_end; p++) {
418                 if ((p->early && parameq(param, p->str)) ||
419                     (strcmp(param, "console") == 0 &&
420                      strcmp(p->str, "earlycon") == 0)
421                 ) {
422                         if (p->setup_func(val) != 0)
423                                 pr_warn("Malformed early option '%s'\n", param);
424                 }
425         }
426         /* We accept everything at this stage. */
427         return 0;
428 }

reference:
[1] https://danielmaker.github.io/blog/linux/kernel_parameter_parsing.html
[2] http://blog.csdn.net/goto_chen/article/details/17392245
[3] http://blog.csdn.net/skyflying2012/article/details/41142801




Fix-Mapped Linear Address

Architecture: ARM64
[Intro]
Fixmap是固定一直指到physical addr的特定位址
Kernel會一次配一塊4K的virtual memory mapping 到physical addr

[Usage of Fix-Mapped Linear Address]
Kernel linear address的第四個GB中至少會有一塊128MB的memory mapping到physical address
這一塊memory 可以讓kernel implement 
(1) noncontiguous memory allocation (2) fix-mapped linear address

[Fix-Mapped Linear Address VS. Physical Address]
Fix-Mapped Linear Address 是一個constant address. 例如 0xffffc000
每一個Fix-Mapped Linear Address maps到一個page frame(4K) 的physical memory
Fix-Mapped Linear Address 跟 linear address that map the first 896MB of RAM 很像
但Fix-Mapped Linear Address可以mapping到任何physical address

[Data Structure enum fixed_addresses]
每一個fix-mapped linear address都有一個專用的integer index defined 在 enum fixed_addresses中
 36 enum fixed_addresses {
 37         FIX_HOLE,
 38 
 39         /*
 40          * Reserve a virtual window for the FDT that is 2 MB larger than the
 41          * maximum supported size, and put it at the top of the fixmap region.
 42          * The additional space ensures that any FDT that does not exceed
 43          * MAX_FDT_SIZE can be mapped regardless of whether it crosses any
 44          * 2 MB alignment boundaries.
 45          *
 46          * Keep this at the top so it remains 2 MB aligned.
 47          */
 48 #define FIX_FDT_SIZE            (MAX_FDT_SIZE + SZ_2M)
 49         FIX_FDT_END,
 50         FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
 51 
 52         FIX_EARLYCON_MEM_BASE,
 53         FIX_TEXT_POKE0,
 54         __end_of_permanent_fixed_addresses,
 55 
 56         /*
 57          * Temporary boot-time mappings, used by early_ioremap(),
 58          * before ioremap() is functional.
 59          */
 60 #define NR_FIX_BTMAPS           (SZ_256K / PAGE_SIZE)
 61 #define FIX_BTMAPS_SLOTS        7
 62 #define TOTAL_FIX_BTMAPS        (NR_FIX_BTMAPS * FIX_BTMAPS_SLOTS)
 63 
 64         FIX_BTMAP_END = __end_of_permanent_fixed_addresses,
 65         FIX_BTMAP_BEGIN = FIX_BTMAP_END + TOTAL_FIX_BTMAPS - 1,
 66 
 67         /*
 68          * Used for kernel page table creation, so unmapped memory may be used
 69          * for tables.
 70          */
 71         FIX_PTE,
 72         FIX_PMD,
 73         FIX_PUD,
 74         FIX_PGD,
 75 
 76         __end_of_fixed_addresses
 77 };

[How to Obtain the Linear Address Set of a Fix-Mapped Linear Address]
23 #ifndef __ASSEMBLY__
 24 /*
 25  * 'index to address' translation. If anyone tries to use the idx
 26  * directly without translation, we catch the bug with a NULL-deference
 27  * kernel oops. Illegal ranges of incoming indices are caught too.
 28  */
 29 static __always_inline unsigned long fix_to_virt(const unsigned int idx)
 30 {
 31         BUILD_BUG_ON(idx >= __end_of_fixed_addresses);
 32         return __fix_to_virt(idx);
 33 }

fix_to_virt -> __fix_to_virt: 能夠根據index找到 constant linear address
如下, PAGE_SHIFT = 12
所以每一個index 對應到的linear address是從FIXADDR_TOP開始減 4K 的倍數
20 #define __fix_to_virt(x)        (FIXADDR_TOP - ((x) << PAGE_SHIFT))


目前畫出的memory layout不一定正確, 若有誤會再修正

[Reference]
Fixmap:
http://palliatory66.rssing.com/chan-60693167/latest.php

ARM64 memory
http://blog.csdn.net/qianlong4526888/article/details/9058221

Linux doc about arm64 memory
http://lxr.free-electrons.com/source/Documentation/arm64/memory.txt?v=4.4

Linux kernel memory management
https://0xax.gitbooks.io/linux-insides/content/mm/linux-mm-2.html

Windows7 memory layout
http://www.codemachine.com/article_x64kvas.html


2016年10月24日 星期一

setup_arch(1) - setup_machine_fdt()

We will focus on device tree initialization in this article
First, take a look of setup_arch() -> setup_machine_fdt()
179 static void __init setup_machine_fdt(phys_addr_t dt_phys)
180 {
181         void *dt_virt = fixmap_remap_fdt(dt_phys);
182 
183         if (!dt_virt || !early_init_dt_scan(dt_virt)) {
184                 pr_crit("\n"
185                         "Error: invalid device tree blob at physical address %pa (virtual address 0x%p)\n"
186                         "The dtb must be 8-byte aligned and must not exceed 2 MB in size\n"
187                         "\nPlease check your bootloader.",
188                         &dt_phys, dt_virt);
189 
190                 while (true)
191                         cpu_relax();
192         }
193 
194         dump_stack_set_arch_desc("%s (DT)", of_flat_dt_get_machine_name());
195 }

1. fixmap_remap_fdt(): 找到device tree的virtual addr
2. early_init_dt_scan(dt_virt): 為之後的DTB scan 做準備, 並進行參數傳遞
3. dump_stack_set_arch_desc(): 印出hardware name, 並把stack dump出來
由 2 進一步從early_init_dt_scan -> early_init_dt_scan_nodes() 看
1212 void __init early_init_dt_scan_nodes(void)
1213 {
1214         /* Retrieve various information from the /chosen node */
1215         of_scan_flat_dt(early_init_dt_scan_chosen, boot_command_line);
1216 
1217         /* Initialize {size,address}-cells info */
1218         of_scan_flat_dt(early_init_dt_scan_root, NULL);
1219 
1220         /* Setup memory, calling early_init_dt_add_memory_arch */
1221         of_scan_flat_dt(early_init_dt_scan_memory, NULL);
1222 }

第一個of_scan_flat_dt: scan /chosen node, save bootargs to boot_command_line, 還處理initrd相關的property, 存到initrd_start, initrd_end這兩個global variables中

第二個of_scan_flat_dt: scan root node, 取得 {size, addr}-cells 的資訊, 並存到dt_root_size_cells 和 dt_root_addr_cells global variables中

第三個of_scan_flat_dt: scan DTB中的memory node, 並把相關的資訊存入meminfo, meminfo是global variable, 保存系統的memory相關的資訊





全國推廣動物認領養平台串聯貼紙

全國推廣動物認領養平台串聯貼紙