This patch adds the ARM_LPAE and ARCH_PHYS_ADDR_T_64BIT Kconfig entries
allowing LPAE support to be compiled into the kernel.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Memory banks living outside of the 32-bit physical address
space do not have a 1:1 pa <-> va mapping and therefore the
__va macro may wrap.
This patch ensures that such banks are marked as highmem so
that the Kernel doesn't try to split them up when it sees that
the wrapped virtual address overlaps the vmalloc space.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
With LPAE, the pgd is a separate page table with entries pointing to the
pmd. The identity_mapping_add() function needs to ensure that the pgd is
populated before populating the pmd level. The do..while blocks now loop
over the pmd in order to have the same implementation for the two page
table formats. The pmd_addr_end() definition has been removed and the
generic one used instead. The pmd clean-up is done in the pgd_free()
function.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
With LPAE, TTBRx registers are 64-bit. The ASID is stored in TTBR0
rather than a separate Context ID register. This patch makes the
necessary changes to handle context switching on LPAE.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The DFSR and IFSR register format is different when LPAE is enabled. In
addition, DFSR and IFSR have similar definitions for the fault type.
This modifies the fault code to correctly handle the new format.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Similar to the PTE freeing, this patch introduced __pmd_free_tlb() which
invalidates the TLB before freeing a PMD page. This is needed because on
newer processors the entry in the upper page table may be cached by the
TLB and point to random data after the PMD has been freed.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch adds the MMU initialisation for the LPAE page table format.
The swapper_pg_dir size with LPAE is 5 rather than 4 pages. A new
proc-v7-3level.S file contains the TTB initialisation, context switch
and PTE setting code with the LPAE. The TTBRx split is based on the
PAGE_OFFSET with TTBR1 used for the kernel mappings. The 36-bit mappings
(supersections) and a few other memory types in mmu.c are conditionally
compiled.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch modifies the pgd/pmd/pte manipulation functions to support
the 3-level page table format. Since there is no need for an 'ext'
argument to cpu_set_pte_ext(), this patch conditionally defines a
different prototype for this function when CONFIG_ARM_LPAE.
The patch also introduces the L_PGD_SWAPPER flag to mark pgd entries
pointing to pmd tables pre-allocated in the swapper_pg_dir and avoid
trying to free them at run-time. This flag is 0 with the classic page
table format.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch introduces the pgtable-3level*.h files with definitions
specific to the LPAE page table format (3 levels of page tables).
Each table is 4KB and has 512 64-bit entries. An entry can point to a
40-bit physical address. The young, write and exec software bits share
the corresponding hardware bits (negated). Other software bits use spare
bits in the PTE.
The patch also changes some variable types from unsigned long or int to
pteval_t or pgprot_t.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Before we enable the MMU, we must ensure that the TTBR registers contain
sane values. After the MMU has been enabled, we jump to the *virtual*
address of the following function, so we also need to ensure that the
SCTLR write has taken effect.
This patch adds ISB instructions around the SCTLR write to ensure the
visibility of the above.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch modifies the proc-v7.S file so that it only contains code
shared between classic MMU and LPAE. The non-common code is factored out
into a separate file.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The FSR structure is different with LPAE and this patch moves the
classic MMU specific definition to a separate fsr-2level.c file that is
included in fault.c. It also moves the fsr_fs and FSR bits to the
fault.h file.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The page table maintenance macros need to be duplicated between the
classic and the LPAE MMU so this patch moves those that are not common
to the pgtable-2level.h file.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Nick Piggin noted upon introducing 4level-fixup.h:
| Add a temporary "fallback" header so architectures can run with
| the 4level pagetables patch without modification. All architectures
| should be converted to use the folding headers (include/asm-generic/
| pgtable-nop?d.h) as soon as possible, and the fallback header removed.
This makes ARM compliant with this statement.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
With the arch/arm code conversion to pgtable-nopud.h, the section and
supersection (un|re)map code triggers compiler warnings on UP systems.
This is caused by pmd_offset() being given a pgd_t argument rather than
a pud_t one. This patch makes the necessary conversion with the
assumption that the pud is folded into the pgd. The page table setting
code only loops over the pmd which is enough with the classic page
tables. This code is not compiled when LPAE is enabled.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The ARM SMP booting code allocates a temporary set of page tables
containing an identity mapping of the kernel image and provides this
to secondary CPUs for initial booting.
In reality, we only need to include the __turn_mmu_on function in the
identity mapping since the rest of the kernel is executing from virtual
addresses after this point.
This patch adds __turn_mmu_on to the .idmap.text section, allowing the
SMP booting code to use the idmap_pgd directly and not have to populate
its own set of page table.
As a result of this patch, we can make the identity_mapping_add function
static (since it is only used within mm/idmap.c) and also remove the
identity_mapping_del function. The identity map population is moved to
an early initcall so that it is setup in time for secondary CPU bringup.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
__create_page_tables identity maps the region of memory from
__enable_mmu to the end of __turn_mmu_on.
In preparation for including __turn_mmu_on in the .idmap.text section,
this patch modifies the identity mapping so that it only includes the
__turn_mmu_on code.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
For soft-rebooting a system, it is necessary to map the MMU-off code
with an identity mapping so that execution can continue safely once the
MMU has been switched off.
Currently, switch_mm_for_reboot takes out a 1:1 mapping from 0x0 to
TASK_SIZE during reboot in the hope that the reset code lives at a
physical address corresponding to a userspace virtual address.
This patch modifies the code so that we switch to the idmap_pgd tables,
which contain a 1:1 mapping of the cpu_reset code. This has the
advantage of only remapping the code that we need and also means we
don't need to worry about allocating a pgd from an atomic context in the
case that the physical address of the cpu_reset code aliases with the
virtual space used by the kernel.
Acked-by: Dave Martin <dave.martin@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
The CPU reset functions disable the MMU and therefore must be executed
with an identity mapping in place.
This patch places the CPU reset functions into the .idmap.text section,
causing the idmap code to include them as part of the identity mapping.
Acked-by: Dave Martin <dave.martin@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
The ARM CPU suspend code requires cpu_resume_mmu to be identity mapped
in order to re-enable the MMU when coming out of suspend. Currently,
this is accomplished by maintaining a suspend_pgd with the relevant
mapping put in place at init time.
This patch replaces the use of suspend_pgd with the new idmap_pgd.
cpu_resume_mmu is placed in the .idmap.text section so that it is
included in the identity map.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Dave Martin <dave.martin@linaro.org>
Tested-by: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
When disabling and re-enabling the MMU, it is necessary to take out an
identity mapping for the code that manipulates the SCTLR in order to
avoid it disappearing from under our feet. This is useful when soft
rebooting and returning from CPU suspend.
This patch allocates a set of page tables during boot and populates them
with an identity mapping for the .idmap.text section. This means that
users of the identity map do not need to manage their own pgd and can
instead annotate their functions with __idmap or, in the case of assembly
code, place them in the correct section.
Acked-by: Dave Martin <dave.martin@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Fix the below build break by including common.h
arch/arm/mach-omap2/cpuidle34xx.c: In function 'omap3_enter_idle':
arch/arm/mach-omap2/cpuidle34xx.c:117: error: implicit declaration of function 'omap_irq_pending'
make[1]: *** [arch/arm/mach-omap2/cpuidle34xx.o] Error 1
make: *** [arch/arm/mach-omap2] Error 2
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Fix the build break by adding the necessary irq functions to
common header.
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Trivial fix to fix below build error:
CC arch/arm/mach-exynos/mach-universal_c210.o
arch/arm/mach-exynos/mach-universal_c210.c:24: error: expected identifier or '(' before '<' token
Signed-off-by: Axel Lin <axel.lin@gmail.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Fix below build error:
CC arch/arm/mach-exynos/cpu.o
arch/arm/mach-exynos/cpu.c: In function 'exynos4_init_irq':
arch/arm/mach-exynos/cpu.c:245: error: 'gic_bank_offset' undeclared (first use in this function)
arch/arm/mach-exynos/cpu.c:245: error: (Each undeclared identifier is reported only once
arch/arm/mach-exynos/cpu.c:245: error: for each function it appears in.)
arch/arm/mach-exynos/cpu.c:243: warning: unused variable 'bank_offset'
make[1]: *** [arch/arm/mach-exynos/cpu.o] Error 1
make: *** [arch/arm/mach-exynos] Error 2
Signed-off-by: Axel Lin <axel.lin@gmail.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Conflicts:
arch/arm/mach-omap2/board-4430sdp.c
arch/arm/mach-omap2/board-omap4panda.c
arch/arm/mach-omap2/include/mach/omap4-common.h
arch/arm/plat-omap/include/plat/irqs.h
The changes to omap4-common.h were moved to arch/arm/mach-omap2/common.h
and the other trivial conflicts resolved. The now empty ifdef in irqs.h
was also eliminated.
The poodle restart handler was added in 74617fb6b8 (3593/1: Add reboot
and shutdown handlers for Zaurus handhelds), and at that time it was
necessary to deal with the RCSR register. This commit also forced all
restarts to use the 'hard' restart method.
In dc38e2ad53 (pxa: Fix RCSR handling), the RCSR handling was removed,
leaving just the forcing to use a 'hard' restart. As hard restarts are
the default (in the absense of a reboot= command line argument), this
seems pointless. In any case, Richard Purdie says:
> From what I remember that hardware either always reboots or always
> halts. I think the option was therefore left hardcoded to make it clear
> it wasn't expected to work. Later Zaurii models could do either but
> required some manual poking of registers to make it happen iirc.
>
> Regardless, you can probably clean this up as you suggest now.
So, lets remove the unnecessary interception and rely on the default
restart mode.
Acked-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Rather than having separate extern declarations in nuc9*.h, provide
a common header file containing these definitions.
Tested-by: Wan Zongshun <mcuos.com@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
We only need to set the system up for a soft-restart if we're going to
be doing a soft-restart. Provide a new function (soft_restart()) which
does the setup and final call for this, and make platforms use it.
Eliminate the call to setup_restart() from the default handler.
This means that platforms arch_reset() function is no longer called with
the page tables prepared for a soft-restart, and caches will still be
enabled.
Acked-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: H Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: Kukjin Kim <kgene.kim@samsung.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Acked-by: Krzysztof Ha■asa <khc@pm.waw.pl>
Acked-by: Paul Mundt <lethal@linux-sh.org>
Acked-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Acked-by: Wan ZongShun <mcuos.com@gmail.com>
Acked-by: Eric Miao <eric.y.miao@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
The separate files for clps711x which contain maybe one or two data
structures or functions is an inefficient use of files, and encourages
further small files as other changes happen. Collapse down these into
a single core.c file.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
As suggested by Russell King - ARM Linux <linux@arm.linux.org.uk>,
there's no need to keep local prototypes in non-local headers.
Add mach-omap1/common.h and mach-omap2/common.h and move the
local prototypes there from plat/common.h and mach/omap4-common.h.
Signed-off-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
IRQs are already disabled by the time arch_reset() is called, so these
calls to local_irq_disable() instead arch_reset() are redundant. Remove
them.
Acked-by: H Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Now that all platforms are converted to MULTI_IRQ_HANDLER, remove the
legacy support.
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for picoxcell to help building multi platform kernels.
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for samsung to help building multi platform kernels.
Cc: Kukjin Kim <kgene.kim@samsung.com>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for versatile to help building multi platform kernels.
Cc: Russell King <linux@arm.linux.org.uk>
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for u300 to help building multi platform kernels.
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for spear to help building multi platform kernels.
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Cc: Rajeev Kumar <rajeev-dlh.kumar@st.com>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for s3c64xx to help building multi platform kernels.
Cc: Ben Dooks <ben-linux@fluff.org>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for nomadik to help building multi platform kernels.
Cc: Alessandro Rubini <rubini@unipv.it>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Cc: STEricsson <STEricsson_nomadik_linux@list.st.com>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for netx to help building multi platform kernels.
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that there is a generic IRQ handler for multiple VIC devices use it
for ep93xx to help building multi platform kernels.
Cc: Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Ryan Mallon <rmallon@gmail.com>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Add a handler for the VIC that is suitable for MULTI_IRQ_HANDLER
platforms. This can replace the ASM entry macros for platforms that use
the VIC.
v4: - rebase ontop of move __exception and friends to
asm/exception.h
- rework polling loop to handle as many irqs as possible in one go
v3: - simplify irq handling loop as suggested by Grant
- service interrupts from msb->lsb order
v2: - allow the handler be used for !CONFIG_OF
- use irq_domain_to_irq()
Cc: Rob Herring <robherring2@gmail.com>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
This adds a device tree binding for the VIC based on the of_irq_init()
support. This adds an irqdomain to the vic and always registers all
vics in the static vic array rather than for pm only to keep track of
the irq domain. struct irq_data::hwirq is used where appropriate rather
than runtime masking.
v3: - include linux/export.h for THIS_MODULE
v2: - use irq_domain_simple_ops
- remove stub implementation of vic_of_init for !CONFIG_OF
- Make VIC select IRQ_DOMAIN
Reviewed-by: Rob Herring <robherring2@gmail.com>
Reviewed-by: Grant Likely <grant.likely@secretlab.ca>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Now that MULTI_IRQ_HANDLER is selected by all the in-tree
GIC users, make it mandatory and remove the unused macros.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Convert the omap2plus platforms to be using CONFIG_MULTI_IRQ_HANDLER.
Each machine is modified to provide either omap2_intc_handle_irq(),
omap3_intc_handle_irq() or gic_handle_irq().
This allows for a major cleanup, removing the MULTI_OMAP setup
from the interrupt path.
Tested on both Panda and IGEPv2 (single kernel image)
Tested-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide the OMAP2/3 IRQ code with low level handlers that can be used
by platforms using CONFIG_MULTI_IRQ_HANDLER. Though the handlers are
written in C, the compiled code looks very similar to its assembly
counterpart (at least with my gcc 4.4.1).
Tested-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
