Commit graph

101 commits

Author SHA1 Message Date
Catalin Marinas
4e7682d077 ARM: 7301/1: Rename the T() macro to TUSER() to avoid namespace conflicts
This macro is used to generate unprivileged accesses (LDRT/STRT) to user
space.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2012-01-25 11:07:40 +00:00
Will Deacon
2d81f1fe81 ARM: lib: add call_with_stack function for safely changing stack
When disabling the MMU, it is necessary to take out a 1:1 identity map
of the reset code so that it can safely be executed with and without
the MMU active. To avoid the situation where the physical address of the
reset code aliases with the virtual address of the active stack (which
cannot be included in the 1:1 mapping), it is desirable to change to a
new stack at a location which is less likely to alias.

This code adds a new lib function, call_with_stack:

	void call_with_stack(void (*fn)(void *), void *arg, void *sp);

which changes the stack to point at the sp parameter, before invoking
fn(arg) with the new stack selected.

Reviewed-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Dave Martin <dave.martin@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2011-12-12 16:07:35 +00:00
Will Deacon
c36ef4b176 ARM: 7171/1: unwind: add unwind directives to bitops assembly macros
The bitops functions (e.g. _test_and_set_bit) on ARM do not have unwind
annotations and therefore the kernel cannot backtrace out of them on a
fatal error (for example, NULL pointer dereference).

This patch annotates the bitops assembly macros with UNWIND annotations
so that we can produce a meaningful backtrace on error. Callers of the
macros are modified to pass their function name as a macro parameter,
enforcing that the macros are used as standalone function implementations.

Acked-by: Dave Martin <dave.martin@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-11-26 21:58:53 +00:00
Russell King
bdf4e94823 Merge branch 'misc' into for-linus
Conflicts:
	arch/arm/mach-integrator/integrator_ap.c
2011-10-25 08:19:59 +01:00
Laura Abbott
01885bc5ce ARM: 7125/1: Add unwinding annotations for 64bit division functions
The 64bit division functions never had unwinding annotations
added. This prevents a backtrace from being printed within
the function and if a division by 0 occurs. Add the annotations.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-10-17 09:13:42 +01:00
Laura Abbott
b380ab4f85 ARM: 7068/1: process: change from __backtrace to dump_stack in show_regs
Currently, show_regs calls __backtrace which does
nothing if CONFIG_FRAME_POINTER is not set. Switch to
dump_stack which handles both CONFIG_FRAME_POINTER and
CONFIG_ARM_UNWIND correctly.

__backtrace is now superseded by dump_stack in general
and show_regs was the last caller so remove __backtrace
as well.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-10-17 09:12:41 +01:00
Arnd Bergmann
7816e210a7 ARM: include linux/highmem.h in uaccess functions
When highpte support is enabled, this is required to build
the kernel.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2011-10-02 15:44:32 +02:00
Linus Torvalds
4d4487140d arm: remove "optimized" SHA1 routines
Since commit 1eb19a12bd ("lib/sha1: use the git implementation of
SHA-1"), the ARM SHA1 routines no longer work.  The reason? They
depended on the larger 320-byte workspace, and now the sha1 workspace is
just 16 words (64 bytes).  So the assembly version would overwrite the
stack randomly.

The optimized asm version is also probably slower than the new improved
C version, so there's no reason to keep it around.  At least that was
the case in git, where what appears to be the same assembly language
version was removed two years ago because the optimized C BLK_SHA1 code
was faster.

Reported-and-tested-by: Joachim Eastwood <manabian@gmail.com>
Cc: Andreas Schwab <schwab@linux-m68k.org>
Cc: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-08-07 14:07:03 -07:00
Rob Herring
b480a4b0c8 ARM: remove unnecessary mach/hardware.h includes
Remove some includes of mach/hardware.h which are not needed. hardware.h
will be removed completely for tegra and cns3xxx in follow on patch.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
2011-07-12 11:19:27 -05:00
Laura Abbott
81479c246c ARM: 6945/1: Add unwinding support for division functions
The software division functions never had unwinding annotations
added. Currently, when a division by zero occurs the backtrace shown
will stop at Ldiv0 or some completely unrelated function. Add
unwinding annotations in hopes of getting a more useful backtrace
when a division by zero occurs.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Acked-by: Dave Martin <dave.martin@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-05-27 22:56:53 +01:00
Russell King
196f020fbb Merge branches 'fixes', 'pgt-next' and 'versatile' into devel 2011-03-20 09:32:12 +00:00
Russell King
516295e5ab ARM: pgtable: add pud-level code
Add pud_offset() et.al. between the pgd and pmd code in preparation of
using pgtable-nopud.h rather than 4level-fixup.h.

This incorporates a fix from Jamie Iles <jamie@jamieiles.com> for
uaccess_with_memcpy.c.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-02-21 19:24:14 +00:00
Dave Martin
3ba6e69ad8 ARM: 6653/1: bitops: Use BX instead of MOV PC,LR
The kernel doesn't officially need to interwork, but using BX
wherever appropriate will help educate people into good assembler
coding habits.

BX is appropriate here because this code is predicated on
__LINUX_ARM_ARCH__ >= 6

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-02-19 16:07:21 +00:00
Russell King
6323f0cced ARM: bitops: switch set/clear/change bitops to use ldrex/strex
Switch the set/clear/change bitops to use the word-based exclusive
operations, which are only present in a wider range of ARM architectures
than the byte-based exclusive operations.

Tested record:
- Nicolas Pitre: ext3,rw,le
- Sourav Poddar: nfs,le
- Will Deacon: ext3,rw,le
- Tony Lindgren: ext3+nfs,le

Reviewed-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: Sourav Poddar <sourav.poddar@ti.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Tested-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-02-02 21:23:25 +00:00
Russell King
a16ede35a2 ARM: bitops: ensure set/clear/change bitops take a word-aligned pointer
Add additional instructions to our assembly bitops functions to ensure
that they only operate on word-aligned pointers.  This will be necessary
when we switch these operations to use the word-based exclusive
operations.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-02-02 21:21:53 +00:00
Russell King
56949d414a ARM: udelay: prevent math rounding resulting in short udelays
We perform the microseconds to loops calculation using a number of
multiplies and shift rights.  Each shift right rounds down the
resulting value, which can result in delays shorter than requested.
Ensure that we always round up.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-01-10 23:55:59 +00:00
Russell King
4ec3eb1363 Merge branch 'smp' into misc
Conflicts:
	arch/arm/kernel/entry-armv.S
	arch/arm/mm/ioremap.c
2011-01-06 22:32:03 +00:00
James Jones
0e91ec0c06 ARM: 6482/2: Fix find_next_zero_bit and related assembly
The find_next_bit, find_first_bit, find_next_zero_bit
and find_first_zero_bit functions were not properly
clamping to the maxbit argument at the bit level. They
were instead only checking maxbit at the byte level.
To fix this, add a compare and a conditional move
instruction to the end of the common bit-within-the-
byte code used by all the functions and be sure not to
clobber the maxbit argument before it is used.

Cc: <stable@kernel.org>
Reviewed-by: Nicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: James Jones <jajones@nvidia.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-24 20:17:46 +00:00
Catalin Marinas
247055aa21 ARM: 6384/1: Remove the domain switching on ARMv6k/v7 CPUs
This patch removes the domain switching functionality via the set_fs and
__switch_to functions on cores that have a TLS register.

Currently, the ioremap and vmalloc areas share the same level 1 page
tables and therefore have the same domain (DOMAIN_KERNEL). When the
kernel domain is modified from Client to Manager (via the __set_fs or in
the __switch_to function), the XN (eXecute Never) bit is overridden and
newer CPUs can speculatively prefetch the ioremap'ed memory.

Linux performs the kernel domain switching to allow user-specific
functions (copy_to/from_user, get/put_user etc.) to access kernel
memory. In order for these functions to work with the kernel domain set
to Client, the patch modifies the LDRT/STRT and related instructions to
the LDR/STR ones.

The user pages access rights are also modified for kernel read-only
access rather than read/write so that the copy-on-write mechanism still
works. CPU_USE_DOMAINS gets disabled only if the hardware has a TLS register
(CPU_32v6K is defined) since writing the TLS value to the high vectors page
isn't possible.

The user addresses passed to the kernel are checked by the access_ok()
function so that they do not point to the kernel space.

Tested-by: Anton Vorontsov <cbouatmailru@gmail.com>
Cc: Tony Lindgren <tony@atomide.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-04 15:44:31 +00:00
Linus Torvalds
be82ae0238 Merge branch 'devel' of master.kernel.org:/home/rmk/linux-2.6-arm
* 'devel' of master.kernel.org:/home/rmk/linux-2.6-arm: (291 commits)
  ARM: AMBA: Add pclk support to AMBA bus infrastructure
  ARM: 6278/2: fix regression in RealView after the introduction of pclk
  ARM: 6277/1: mach-shmobile: Allow users to select HZ, default to 128
  ARM: 6276/1: mach-shmobile: remove duplicate NR_IRQS_LEGACY
  ARM: 6246/1: mmci: support larger MMCIDATALENGTH register
  ARM: 6245/1: mmci: enable hardware flow control on Ux500 variants
  ARM: 6244/1: mmci: add variant data and default MCICLOCK support
  ARM: 6243/1: mmci: pass power_mode to the translate_vdd callback
  ARM: 6274/1: add global control registers definition header file for nuc900
  mx2_camera: fix type of dma buffer virtual address pointer
  mx2_camera: Add soc_camera support for i.MX25/i.MX27
  arm/imx/gpio: add spinlock protection
  ARM: Add support for the LPC32XX arch
  ARM: LPC32XX: Arch config menu supoport and makefiles
  ARM: LPC32XX: Phytec 3250 platform support
  ARM: LPC32XX: Misc support functions
  ARM: LPC32XX: Serial support code
  ARM: LPC32XX: System suspend support
  ARM: LPC32XX: GPIO, timer, and IRQ drivers
  ARM: LPC32XX: Clock driver
  ...
2010-08-03 14:31:24 -07:00
Russell King
4609a179c9 ARM: Fix csum_partial_copy_from_user()
Using the parent functions frame pointer to access our arguments is
completely wrong, whether or not we're building with frame pointers
or not.  What we should be using is the stack pointer to get at the
word above the registers we stacked ourselves.

Reported-by: Bosko Radivojevic <bosko.radivojevic@gmail.com>
Tested-by: Bosko Radivojevic <bosko.radivojevic@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-07-26 12:18:16 +01:00
Russell King
c9c6fe5033 ARM: Remove support for LinkUp Systems L7200 SDP.
This hasn't been actively maintained for a long time, only receiving
the occasional build update when things break.  I doubt anyone has
one of these on their desks anymore.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-06-24 15:41:31 +01:00
Catalin Marinas
e814d826dc ARM: 6110/1: Fix Thumb-2 kernel builds when UACCESS_WITH_MEMCPY is enabled
The patch adds the ENDPROC declarations for the __copy_to_user_std and
__clear_user_std functions. Without these, the compiler generates BXL to
ARM when compiling the kernel in Thumb-2 mode.

Reported-by: Kyungmin Park <kmpark@infradead.org>
Tested-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-05-08 10:45:26 +01:00
Russell King
4260415f6a ARM: fix build error in arch/arm/kernel/process.c
/tmp/ccJ3ssZW.s: Assembler messages:
/tmp/ccJ3ssZW.s:1952: Error: can't resolve `.text' {.text section} - `.LFB1077'

This is caused because:

	.section .data
	.section .text
	.section .text
	.previous

does not return us to the .text section, but the .data section; this
makes use of .previous dangerous if the ordering of previous sections
is not known.

Fix up the other users of .previous; .pushsection and .popsection are
a safer pairing to use than .section and .previous.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-04-21 08:45:21 +01:00
Tejun Heo
336f5899d2 Merge branch 'master' into export-slabh 2010-04-05 11:37:28 +09:00
Tejun Heo
5a0e3ad6af include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-30 22:02:32 +09:00
Catalin Marinas
fd522a8dec ARM: 6006/1: ARM: Use the correct NOP size in memmove for Thumb-2 kernel builds
When compiling the kernel to Thumb-2, using a 16-bit NOP in the
memmove() implementation causes the preceding ADD PC instruction to
branch incorrectly in the middle of a 32-bit LDR or STR instruction. The
memmove() code is now similar to the memcpy() template.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-03-29 17:33:33 +01:00
Russell King
40d743b8c1 Merge branch 'for-rmk' of git://linux-arm.org/linux-2.6 2009-09-19 13:47:57 +01:00
Kirill A. Shutemov
dca230f00d ARM: 5701/1: ARM: copy_page.S: take into account the size of the cache line
Optimized version of copy_page() was written with assumption that cache
line size is 32 bytes. On Cortex-A8 cache line size is 64 bytes.

This patch tries to generalize copy_page() to work with any cache line
size if cache line size is multiple of 16 and page size is multiple of
two cache line size.

After this optimization we've got ~25% speedup on OMAP3(tested in
userspace).

There is test for kernelspace which trigger copy-on-write after fork():

 #include <stdlib.h>
 #include <string.h>
 #include <unistd.h>

 #define BUF_SIZE (10000*4096)
 #define NFORK 200

 int main(int argc, char **argv)
 {
         char *buf = malloc(BUF_SIZE);
         int i;

         memset(buf, 0, BUF_SIZE);

         for(i = 0; i < NFORK; i++) {
                 if (fork()) {
                         wait(NULL);
                 } else {
                         int j;

                         for(j = 0; j < BUF_SIZE; j+= 4096)
                                 buf[j] = (j & 0xFF) + 1;
                         break;
                 }
         }

         free(buf);
         return 0;
 }

Before optimization this test takes ~66 seconds, after optimization
takes ~56 seconds.

Signed-off-by: Siarhei Siamashka <siarhei.siamashka@nokia.com>
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2009-09-15 22:07:02 +01:00
Nicolas Pitre
2f82af08fc Nicolas Pitre has a new email address
Due to problems at cam.org, my nico@cam.org email address is no longer
valid.  FRom now on, nico@fluxnic.net should be used instead.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-09-15 09:37:12 -07:00
Russell King
9b2616c2e8 Merge branch 'for-rmk-2.6.32' of git://git.pengutronix.de/git/ukl/linux-2.6 into devel-stable 2009-08-15 16:51:48 +01:00
Uwe Kleine-König
0d928b0b61 Complete irq tracing support for ARM
Before this patch enabling and disabling irqs in assembler code and by
the hardware wasn't tracked completly.

I had to transpose two instructions in arch/arm/lib/bitops.h because
restore_irqs doesn't preserve the flags with CONFIG_TRACE_IRQFLAGS=y

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2009-08-13 20:34:37 +02:00
Catalin Marinas
8b592783a2 Thumb-2: Implement the unified arch/arm/lib functions
This patch adds the ARM/Thumb-2 unified support for the arch/arm/lib/*
files.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 12:32:57 +01:00
Catalin Marinas
88987ef91b Thumb-2: Add some .align statements to the .S files
Since the Thumb-2 instructions can be 16-bit wide, data in the .text
sections may not be aligned to a 32-bit word and this leads to unaligned
exceptions. This patch does not affect the ARM code generation.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 12:32:52 +01:00
Russell King
98797a241e Merge branch 'copy_user' of git://git.marvell.com/orion into devel 2009-06-14 10:59:32 +01:00
Nicolas Pitre
c626e3f5ca [ARM] alternative copy_to_user: more precise fallback threshold
Previous size thresholds were guessed from various user space benchmarks
using a kernel with and without the alternative uaccess option.  This
is however not as precise as a kernel based test to measure the real
speed of each method.

This adds a simple test bench to show the time needed for each method.
With this, the optimal size treshold for the alternative implementation
can be determined with more confidence.  It appears that the optimal
threshold for both copy_to_user and clear_user is around 64 bytes. This
is not a surprise knowing that the memcpy and memset implementations
need at least 64 bytes to achieve maximum throughput.

One might suggest that such test be used to determine the optimal
threshold at run time instead, but results are near enough to 64 on
tested targets concerned by this alternative copy_to_user implementation,
so adding some overhead associated with a variable threshold is probably
not worth it for now.

Signed-off-by: Nicolas Pitre <nico@marvell.com>
2009-05-30 01:10:15 -04:00
Nicolas Pitre
cb9dc92c0a [ARM] lower overhead with alternative copy_to_user for small copies
Because the alternate copy_to_user implementation has a higher setup cost
than the standard implementation, the size of the memory area to copy
is tested and the standard implementation invoked instead when that size
is too small.  Still, that test is made after the processor has preserved
a bunch of registers on the stack which have to be reloaded right away
needlessly in that case, causing a measurable performance regression
compared to plain usage of the standard implementation only.

To make the size test overhead negligible, let's factorize it out of
the alternate copy_to_user function where it is clear to the compiler
that no stack frame is needed.  Thanks to CONFIG_ARM_UNWIND allowing
for frame pointers to be disabled and tail call optimization to kick in,
the overhead in the small copy case becomes only 3 assembly instructions.

A similar trick is applied to clear_user as well.

Signed-off-by: Nicolas Pitre <nico@marvell.com>
2009-05-29 22:38:33 -04:00
Lennert Buytenhek
39ec58f3fe [ARM] alternative copy_to_user/clear_user implementation
This implements {copy_to,clear}_user() by faulting in the userland
pages and then using the regular kernel mem{cpy,set}() to copy the
data (while holding the page table lock).  This is a win if the regular
mem{cpy,set}() implementations are faster than the user copy functions,
which is the case e.g. on Feroceon, where 8-word STMs (which memcpy()
uses under the right conditions) give significantly higher memory write
throughput than a sequence of individual 32bit stores.

Here are numbers for page sized buffers on some Feroceon cores:

 - copy_to_user on Orion5x goes from 51 MB/s to 83 MB/s
 - clear_user on Orion5x goes from 89MB/s to 314MB/s
 - copy_to_user on Kirkwood goes from 240 MB/s to 356 MB/s
 - clear_user on Kirkwood goes from 367 MB/s to 1108 MB/s
 - copy_to_user on Disco-Duo goes from 248 MB/s to 398 MB/s
 - clear_user on Disco-Duo goes from 328 MB/s to 1741 MB/s

Because the setup cost is non negligible, this is worthwhile only if
the amount of data to copy is large enough.  The operation falls back
to the standard implementation when the amount of data is below a certain
threshold. This threshold was determined empirically, however some targets
could benefit from a lower runtime determined value for optimal results
eventually.

In the copy_from_user() case, this technique does not provide any
worthwhile performance gain due to the fact that any kind of read access
allocates the cache and subsequent 32bit loads are just as fast as the
equivalent 8-word LDM.

Signed-off-by: Lennert Buytenhek <buytenh@marvell.com>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Tested-by: Martin Michlmayr <tbm@cyrius.com>
2009-05-29 22:36:45 -04:00
Nicolas Pitre
a1f98849fd [ARM] allow for alternative __copy_to_user/__clear_user implementations
This allows for optional alternative implementations of __copy_to_user
and __clear_user, with a possible runtime fallback to the standard
version when the alternative provides no gain over that standard
version. This is done by making the standard __copy_to_user into a weak
alias for the symbol __copy_to_user_std.  Same thing for __clear_user.

Those two functions are particularly good candidates to have alternative
implementations for, since they rely on the STRT instruction which has
lower performances than STM instructions on some CPU cores such as
the ARM1176 and Marvell Feroceon.

Signed-off-by: Nicolas Pitre <nico@marvell.com>
2009-05-29 22:34:45 -04:00
Russell King
bac4e960b5 [ARM] barriers: improve xchg, bitops and atomic SMP barriers
Mathieu Desnoyers pointed out that the ARM barriers were lacking:

- cmpxchg, xchg and atomic add return need memory barriers on
  architectures which can reorder the relative order in which memory
  read/writes can be seen between CPUs, which seems to include recent
  ARM architectures. Those barriers are currently missing on ARM.

- test_and_xxx_bit were missing SMP barriers.

So put these barriers in.  Provide separate atomic_add/atomic_sub
operations which do not require barriers.

Reported-Reviewed-and-Acked-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2009-05-28 19:39:27 +01:00
Russell King
31bccbf392 Merge branch 'clps7500' into devel
Conflicts:

	arch/arm/Kconfig
2008-11-27 12:39:43 +00:00
Russell King
635f0258e5 [ARM] clps7500: remove support
The CLPS7500 platform has not built since 2.6.22-git7 and there
seems to be no interest in fixing it.  So, remove the platform
support.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-11-27 12:38:11 +00:00
Russell King
59f0cb0fdd [ARM] remove memzero()
As suggested by Andrew Morton, remove memzero() - it's not supported
on other architectures so use of it is a potential build breaking bug.
Since the compiler optimizes memset(x,0,n) to __memzero() perfectly
well, we don't miss out on the underlying benefits of memzero().

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-11-27 12:37:59 +00:00
Russell King
6a4690c22f Merge branch 'ptebits' into devel
Conflicts:

	arch/arm/Kconfig
2008-10-09 21:31:56 +01:00
Catalin Marinas
22acc4e650 [ARM] 5231/1: Do not save the frame pointer in the csum_partial_copy_* functions
Since the other assembly functions do not seem to save the frame
pointer onto the stack, this patch changes the csum_partial_copy_*
functions to behave in the same way.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-01 12:06:35 +01:00
Catalin Marinas
9c23e5fefa [ARM] 5232/1: Do not post-index STRT instruction in clear_user.S
The last strnebt instruction has a post-index of 1 but the address
register is set to 0 in the next instruction, so no need for
post-indexing.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-01 12:06:34 +01:00
Catalin Marinas
93ed397011 [ARM] 5227/1: Add the ENDPROC declarations to the .S files
This declaration specifies the "function" type and size for various
assembly functions, mainly needed for generating the correct branch
instructions in Thumb-2.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-01 12:06:34 +01:00
Jean-Christophe DUBOIS
212496fd9a [ARM] 5226/1: remove unmatched comment end.
remove unmatched comment end.

Signed-off-by: Jean-Christophe DUBOIS <jcd@tribudubois.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-08-28 17:12:29 +01:00
Russell King
a09e64fbc0 [ARM] Move include/asm-arm/arch-* to arch/arm/*/include/mach
This just leaves include/asm-arm/plat-* to deal with.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-08-07 09:55:48 +01:00
Russell King
be50972935 [ARM] Remove asm/hardware.h, use asm/arch/hardware.h instead
Remove includes of asm/hardware.h in addition to asm/arch/hardware.h.
Then, since asm/hardware.h only exists to include asm/arch/hardware.h,
update everything to directly include asm/arch/hardware.h and remove
asm/hardware.h.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-08-07 09:40:08 +01:00