This patch avoids copy of buffered data to hash from bufnext to buf
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 doesn't support S/G in descriptors so for hash operations,
the CPU has to build a buffer containing the buffered block and
the incoming data. This generates a lot of memory copies which
represents more than 50% of CPU time of a md5sum operation as
shown below with a 'perf record'.
|--86.24%-- kcapi_md_digest
| |
| |--86.18%-- _kcapi_common_vmsplice_chunk_fd
| | |
| | |--83.68%-- splice
| | | |
| | | |--83.59%-- ret_from_syscall
| | | | |
| | | | |--83.52%-- sys_splice
| | | | | |
| | | | | |--83.49%-- splice_from_pipe
| | | | | | |
| | | | | | |--83.04%-- __splice_from_pipe
| | | | | | | |
| | | | | | | |--80.67%-- pipe_to_sendpage
| | | | | | | | |
| | | | | | | | |--78.25%-- hash_sendpage
| | | | | | | | | |
| | | | | | | | | |--60.08%-- ahash_process_req
| | | | | | | | | | |
| | | | | | | | | | |--56.36%-- sg_copy_buffer
| | | | | | | | | | | |
| | | | | | | | | | | |--55.29%-- memcpy
| | | | | | | | | | | |
However, unlike SEC2+, SEC1 offers the possibility to chain
descriptors. It is therefore possible to build a first descriptor
pointing to the buffered data and a second descriptor pointing to
the incoming data, hence avoiding the memory copy to a single
buffer.
With this patch, the time necessary for a md5sum on a 90Mbytes file
is approximately 3 seconds. Without the patch it takes 6 seconds.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
At every request, we map and unmap the same hash hw_context.
This patch moves the dma mapping/unmapping in functions ahash_init()
and ahash_import().
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
dma_map_single() is an heavy operation which doesn't need to
be done at each request as the key doesn't change.
Instead of DMA mapping the key at every request, this patch maps it
once in setkey()
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Do (desc->hdr & DESC_HDR_TYPE_IPSEC_ESP) only once.
Limit number of if/else paths
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
to_talitos_ptr() and to_talitos_ptr_len() are always called together
in order to fully set a ptr, so lets merge them into a single
helper.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The number of channels is known from the beginning, no need to
test it everytime.
This patch defines two additional done functions handling only channel 0.
Then the probe registers the correct one based on the number of channels.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use of_property_read_u32() to simplify DT read
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
talitos_handle_buggy_hash() and talitos_sg_map() are only used
locally, make them static
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch zeroize the descriptor at allocation using memset().
This has two advantages:
- It reduces the number of places where data has to be set to 0
- It avoids reading memory and loading the cache with data that
will be entirely replaced.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
sg_link_tbl_len shall be used instead of cryptlen, otherwise
SECs which perform HW CICV verification will fail.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
sha224 AEAD test fails with:
[ 2.803125] talitos ff020000.crypto: DEUISR 0x00000000_00000000
[ 2.808743] talitos ff020000.crypto: MDEUISR 0x80100000_00000000
[ 2.814678] talitos ff020000.crypto: DESCBUF 0x20731f21_00000018
[ 2.820616] talitos ff020000.crypto: DESCBUF 0x0628d64c_00000010
[ 2.826554] talitos ff020000.crypto: DESCBUF 0x0631005c_00000018
[ 2.832492] talitos ff020000.crypto: DESCBUF 0x0628d664_00000008
[ 2.838430] talitos ff020000.crypto: DESCBUF 0x061b13a0_00000080
[ 2.844369] talitos ff020000.crypto: DESCBUF 0x0631006c_00000080
[ 2.850307] talitos ff020000.crypto: DESCBUF 0x0631006c_00000018
[ 2.856245] talitos ff020000.crypto: DESCBUF 0x063100ec_00000000
[ 2.884972] talitos ff020000.crypto: failed to reset channel 0
[ 2.890503] talitos ff020000.crypto: done overflow, internal time out, or rngu error: ISR 0x20000000_00020000
[ 2.900652] alg: aead: encryption failed on test 1 for authenc-hmac-sha224-cbc-3des-talitos: ret=22
This is due to SHA224 not being supported by the HW. Allthough for
hash we are able to init the hash context by SW, it is not
possible for AEAD. Therefore SHA224 AEAD has to be deactivated.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Crypto manager test report the following failures:
[ 3.061081] alg: skcipher: setkey failed on test 5 for ecb-des-talitos: flags=100
[ 3.069342] alg: skcipher-ddst: setkey failed on test 5 for ecb-des-talitos: flags=100
[ 3.077754] alg: skcipher-ddst: setkey failed on test 5 for ecb-des-talitos: flags=100
This is due to setkey being expected to detect weak keys.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Today, md5sum fails with error -ENOKEY because a setkey
function is set for non hmac hashing algs, see strace output below:
mmap(NULL, 378880, PROT_READ, MAP_SHARED, 6, 0) = 0x77f50000
accept(3, 0, NULL) = 7
vmsplice(5, [{"bin/\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 378880}], 1, SPLICE_F_MORE|SPLICE_F_GIFT) = 262144
splice(4, NULL, 7, NULL, 262144, SPLICE_F_MORE) = -1 ENOKEY (Required key not available)
write(2, "Generation of hash for file kcap"..., 50) = 50
munmap(0x77f50000, 378880) = 0
This patch ensures that setkey() function is set only
for hmac hashing.
Cc: <stable@vger.kernel.org>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
md5sum on some files gives wrong result
Exemple:
With the md5sum from libkcapi:
c15115c05bad51113f81bdaee735dd09 test
With the original md5sum:
bbdf41d80ba7e8b2b7be3a0772be76cb test
This patch fixes this issue
Cc: <stable@vger.kernel.org>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
An updated patch that also handles the additional key length requirements
for the AEAD algorithms.
The max keysize is not 96. For SHA384/512 it's 128, and for the AEAD
algorithms it's longer still. Extend the max keysize for the
AEAD size for AES256 + HMAC(SHA512).
Cc: <stable@vger.kernel.org> # 3.6+
Fixes: 357fb60502 ("crypto: talitos - add sha224, sha384 and sha512 to existing AEAD algorithms")
Signed-off-by: Martin Hicks <mort@bork.org>
Acked-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Trivial fix to spelling mistake "pointeur" to "pointer"
in dev_err message
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 doesn't have IPSEC_ESP descriptor type but it is able to perform
IPSEC using HMAC_SNOOP_NO_AFEU, which is also existing on SEC2
In order to be able to define descriptors templates for SEC1 without
breaking SEC2+, we have to give lower priority to HMAC_SNOOP_NO_AFEU
so that SEC2+ selects IPSEC_ESP and not HMAC_SNOOP_NO_AFEU which is
less performant.
This is done by adding a priority field in the template. If the field
is 0, we use the default priority, otherwise we used the one in the
field.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patchs enhances the IPSEC_ESP related functions for them to
also supports the same operations with descriptor type
HMAC_SNOOP_NO_AFEU.
The differences between the two descriptor types are:
* pointeurs 2 and 3 are swaped (Confidentiality key and
Primary EU Context IN)
* HMAC_SNOOP_NO_AFEU has CICV out in pointer 6
* HMAC_SNOOP_NO_AFEU has no primary EU context out so we get it
from the end of data out
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In preparation of IPSEC for SEC1, first step is to make the mapping
helpers more generic so that they can also be used by AEAD functions.
First, the functions are moved before IPSEC functions in talitos.c
talitos_sg_unmap() and unmap_sg_talitos_ptr() are merged as they
are quite similar, the second one handling the SEC1 case an calling
the first one for SEC2
map_sg_in_talitos_ptr() and map_sg_out_talitos_ptr() are merged
into talitos_sg_map() and enhenced to support offseted zones
as used for AEAD. The actual mapping is now performed outside that
helper. The DMA sync is also done outside to not make it several
times.
talitos_edesc_alloc() size calculation are fixed to also take into
account AEAD specific parts also for SEC1
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to be able to use the mapping/unmapping helpers for IPSEC
it needs to be move upper in the file
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use helper for all modifications to talitos_ptr in preparation to
the implementation of AEAD for SEC1
to_talitos_ptr_extent_clear() has been removed in favor of
to_talitos_ptr_ext_set() to set any value and
to_talitos_ptr_ext_or() to or the extent field with a value
name has been shorten to help keeping single lines of 80 chars
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Provide hardware state import/export functionality, as mandated by
commit 8996eafdcb ("crypto: ahash - ensure statesize is non-zero")
Cc: <stable@vger.kernel.org> # 4.3+
Reported-by: Jonas Eymann <J.Eymann@gmx.net>
Signed-off-by: Horia Geant? <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
After conversion to new AEAD interface, tcrypt tests fail as follows:
[...]
[ 1.145414] alg: aead: Test 1 failed on encryption for authenc-hmac-sha1-cbc-aes-talitos
[ 1.153564] 00000000: 53 69 6e 67 6c 65 20 62 6c 6f 63 6b 20 6d 73 67
[ 1.160041] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 1.166509] 00000020: 00 00 00 00
[...]
Fix them by providing the correct cipher in & cipher out pointers,
i.e. must skip over associated data in src and dst S/G.
While here, fix a problem with the HW S/G table index usage:
tbl_off must be updated after the pointer to the table entries is set.
Cc: <stable@vger.kernel.org> # 4.3+
Fixes: aeb4c132f3 ("crypto: talitos - Convert to new AEAD interface")
Reported-by: Jonas Eymann <J.Eymann@gmx.net>
Signed-off-by: Horia Geant? <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the following algorithms to the talitos driver:
* ecb(aes)
* ctr(aes)
* ecb(des)
* cbc(des)
* ecb(des3_ede)
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The sg_nents_for_len() function could fail, this patch add a check for
its return value.
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Using non-constant time memcmp() makes the verification of the authentication
tag in the decrypt path vulnerable to timing attacks. Fix this by using
crypto_memneq() instead.
Cc: stable@vger.kernel.org
Signed-off-by: David Gstir <david@sigma-star.at>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The talitos driver use two dma_map_sg path
according to SG are chained or not.
Since dma_map_sg can handle both case, clean the code with all
references to sg chained.
Thus removing talitos_map_sg, talitos_unmap_sg_chain
and sg_count functions.
Signed-off-by: LABBE Corentin <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull SG updates from Jens Axboe:
"This contains a set of scatter-gather related changes/fixes for 4.3:
- Add support for limited chaining of sg tables even for
architectures that do not set ARCH_HAS_SG_CHAIN. From Christoph.
- Add sg chain support to target_rd. From Christoph.
- Fixup open coded sg->page_link in crypto/omap-sham. From
Christoph.
- Fixup open coded crypto ->page_link manipulation. From Dan.
- Also from Dan, automated fixup of manual sg_unmark_end()
manipulations.
- Also from Dan, automated fixup of open coded sg_phys()
implementations.
- From Robert Jarzmik, addition of an sg table splitting helper that
drivers can use"
* 'for-4.3/sg' of git://git.kernel.dk/linux-block:
lib: scatterlist: add sg splitting function
scatterlist: use sg_phys()
crypto/omap-sham: remove an open coded access to ->page_link
scatterlist: remove open coded sg_unmark_end instances
crypto: replace scatterwalk_sg_chain with sg_chain
target/rd: always chain S/G list
scatterlist: allow limited chaining without ARCH_HAS_SG_CHAIN
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
[hch: split from a larger patch by Dan]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Jens Axboe <axboe@fb.com>
The probe error path for this driver, for all intents and purposes,
is the talitos_remove() function due to the common "goto err_out".
Without this patch applied, talitos_remove() will panic under these
two conditions:
1. If the RNG device hasn't been registered via
talitos_register_rng() prior to entry into talitos_remove(),
then the attempt to unregister the RNG "device" will cause a panic.
2. If the priv->chan array has not been allocated prior to entry
into talitos_remove(), then the per-channel FIFO cleanup will panic
because of the dereference of that NULL "array".
Both of the above scenarios occur if talitos_probe_irq() fails.
This patch resolves issue #1 by introducing a boolean to mask the
hwrng_unregister() call in talitos_unregister_rng() if RNG device
registration was unsuccessful.
It resolves issue #2 by checking that priv->chan is not NULL in the
per-channel FIFO cleanup for loop.
Signed-off-by: Aaron Sierra <asierra@xes-inc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts talitos to the new AEAD interface. IV generation
has been removed since it's equivalent to a software implementation.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
-change req_ctx->nbuf from u64 to unsigned int to silence checker
warnings; this is safe since nbuf value is <= HASH_MAX_BLOCK_SIZE
-remove unused value read from TALITOS_CCPSR; there is no requirement
to read upper 32b before reading lower 32b of a 64b register;
SEC RM mentions: "reads can always be done by byte, word, or dword"
-remove unused return value of sg_to_link_tbl()
-change "len" parameter of map_single_talitos_ptr() and
to_talitos_ptr_len() to unsigned int; later, cpu_to_be16 will __force
downcast the value to unsigned short without any checker warning
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Check return value of scatterlist_sg_next(), i.e. don't rely solely
on number of bytes to be processed or number of scatterlist entries.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This reverts commit 7291a932c6.
The conversion to be16_add_cpu() is incorrect in case cryptlen is
negative due to premature (i.e. before addition / subtraction)
implicit conversion of cryptlen (int -> u16) leading to sign loss.
Cc: <stable@vger.kernel.org> # 3.10+
Cc: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Cc: <stable@vger.kernel.org> # 3.2+
Fixes: 1d11911a8c ("crypto: talitos - fix warning: 'alg' may be used uninitialized in this function")
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The + operation has higher precedence than ?: so we need parentheses
here. Otherwise we may end up allocating a max of only one "cryptlen"
instead of two.
Fixes: 6f65f6ac5f ('crypto: talitos - implement scatter/gather copy for SEC1')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We add a specific compatible for SEC1, to handle the differences
between SEC1 and SEC2+
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 bugs on 0 data hash, so we submit an already padded block representing 0 data
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 doesn't support scatter/gather, SEC1 doesn't handle link tables.
Therefore, for SEC1 we have to do it by SW. For that, we reserve
space at the end of the extended descriptor, in lieu of the space
reserved for the link tables on SEC2, and we perform sg_copy() when
preparing the descriptors
We also adapt the max buffer size which is only 32k on SEC1 while it
is 64k on SEC2+
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adapts the interrupts handling and reset function for
SEC1. On SEC1, registers are almost similar to SEC2+, but bits
are sometimes located at different places. So we need to define
TALITOS1 and TALITOS2 versions of some fields, and manage according
to whether it is SEC1 or SEC2.
On SEC1, only one interrupt vector is dedicated to the SEC, so only
interrupt_4ch is needed.
On SEC1, interrupts are enabled by clearing related bits in IMR,
while on SEC2, interrupts are enabled by seting the bits in IMR.
SEC1 also performs parity verification in the DES Unit. We have
to disable this feature because the test vectors provided in
the kernel have parity errors.
In reset functions, only SEC2 supports continuation after error.
For SEC1, we have to reset in all cases.
For errors handling, SEC2+ names have been kept, but displayed
text have been amended to reflect exact meaning on SEC1.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC 1.0, 1.2 and 2.x+ have different EU base addresses, so we need to
define pointers for each EU in the driver private data structure.
The proper address is set by the probe function depending on the
SEC type, in order to provide access to the proper address.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC1 descriptor is a bit different to SEC2+ descriptor.
talitos_submit() will have to copy hdr field into hdr1 field and
send the descriptor starting at hdr1 up to next_desc.
For SEC2, it remains unchanged and next_desc is just ignored.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
talitos descriptor is slightly different for SEC1 and SEC2+, so
lets the helper function that fills the descriptor take into account
the type of SEC.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We add a new feature in the features field, to mark compatible
"fsl,sec1.0"
We also define a helper function called has_ftr_sec1() to help
functions quickly determine if they are running on SEC1 or SEC2+.
When only SEC1 or SEC2 is compiled in, has_ftr_sec1() return
trivial corresponding value. If both are compiled in, feature
field is checked.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a helper function for reads and writes of the len
param of the talitos descriptor. This will help implement
SEC1 later.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
map_single_talitos_ptr() is always called with extent == 0, so lets remove this unused parameter
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
j_extent field is specific to SEC2 so we add a helper function to clear it
so that SEC1 can redefine that function as nop
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Linux CodyingStyle recommends to use short variables for local
variables. ptr is just good enough for those 3 lines functions.
It helps keep single lines shorter than 80 characters.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch refactors the handling of the input and output data that is quite
similar in several functions
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Do use zero_entry value to init the descriptors ptrs to zero instead of
writing 0 in each field
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This is properly defined in the md5 header file.
Signed-off-by: Martin Hicks <mort@bork.org>
Acked-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There were multiple loops in a row, for each separate step of the
initialization of the channels. Simplify to a single loop.
Signed-off-by: Martin Hicks <mort@bork.org>
Acked-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Modify crypto drivers to use the generic SG helper since
both of them are equivalent and the one from crypto is redundant.
See also:
468577abe3 reverted in
b2ab4a57b0
Signed-off-by: Cristian Stoica <cristian.stoica@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In function talitos_probe(), it will jump to err_out when getting an
error in talitos_probe_irq(). Then the uninitialized list head
priv->alg_list will be used in function talitos_remove(). In this case
we would get a call trace like the following. So move up the
initialization of priv->alg_list.
Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xc0459ff4
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=8 P1020 RDB
Modules linked in:
CPU: 1 PID: 1 Comm: swapper/0 Tainted: G W 3.13.0-08789-g54c0a4b46150 #33
task: cf050000 ti: cf04c000 task.ti: cf04c000
NIP: c0459ff4 LR: c0459fd4 CTR: c02f2438
REGS: cf04dcb0 TRAP: 0300 Tainted: G W (3.13.0-08789-g54c0a4b46150)
MSR: 00029000 <CE,EE,ME> CR: 82000028 XER: 20000000
DEAR: 00000000 ESR: 00000000
GPR00: c045ac28 cf04dd60 cf050000 cf2579c0 00021000 00000000 c02f35b0 0000014e
GPR08: c07e702c cf104300 c07e702c 0000014e 22000024 00000000 c0002a3c 00000000
GPR16: 00000000 00000000 00000000 00000000 00000000 00000000 c082e4e0 000000df
GPR24: 00000000 00100100 00200200 cf257a2c cf0efe10 cf2579c0 cf0efe10 00000000
NIP [c0459ff4] talitos_remove+0x3c/0x1c8
LR [c0459fd4] talitos_remove+0x1c/0x1c8
Call Trace:
[cf04dd60] [c07485d8] __func__.13331+0x1241c8/0x1391c0 (unreliable)
[cf04dd90] [c045ac28] talitos_probe+0x244/0x998
[cf04dde0] [c0306a74] platform_drv_probe+0x28/0x68
[cf04ddf0] [c0304d38] really_probe+0x78/0x250
[cf04de10] [c030505c] __driver_attach+0xc8/0xcc
[cf04de30] [c0302e98] bus_for_each_dev+0x6c/0xb8
[cf04de60] [c03043cc] bus_add_driver+0x168/0x220
[cf04de80] [c0305798] driver_register+0x88/0x130
[cf04de90] [c0002458] do_one_initcall+0x14c/0x198
[cf04df00] [c079f904] kernel_init_freeable+0x138/0x1d4
[cf04df30] [c0002a50] kernel_init+0x14/0x124
[cf04df40] [c000ec40] ret_from_kernel_thread+0x5c/0x64
Signed-off-by: Kevin Hao <haokexin@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Driver core sets it to NULL upon probe failure or release.
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Commit 3e721aeb3d
("crypto: talitos - handle descriptor not found in error path")
tried to address the fact that CDPR (Current Descriptor Pointer Register)
is unreliable.
As it turns out, there are still issues in the function detecting the
offending descriptor:
-only 32 bits of the descriptor address are read, however the address is
36-bit - since reset_channel() initializes channels with EAE (extended
address) bit set
-reading CDPR can return zero in cur_desc; when searching the channel
fifo for this address, cur_desc == dma_desc (= 0) case might happen,
leading to an oops when trying to return desc->hdr (desc is zero)
-read channel's .tail only once; the tail is a moving target; use a
local variable for the end of search condition
Signed-off-by: Lei Xu <Lei.Xu@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Tested-by: Kalyani Chowdhury <Kalyani.Chowdhury@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
talitos does not handle well zero-length assoc data. From dmesg:
talitos ffe30000.crypto: master data transfer error
talitos ffe30000.crypto: gather return/length error
Check whether assoc data is provided by inspecting assoclen,
not assoc pointer.
This is needed in order to pass testmgr tests.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull crypto update from Herbert Xu:
- Made x86 ablk_helper generic for ARM
- Phase out chainiv in favour of eseqiv (affects IPsec)
- Fixed aes-cbc IV corruption on s390
- Added constant-time crypto_memneq which replaces memcmp
- Fixed aes-ctr in omap-aes
- Added OMAP3 ROM RNG support
- Add PRNG support for MSM SoC's
- Add and use Job Ring API in caam
- Misc fixes
[ NOTE! This pull request was sent within the merge window, but Herbert
has some questionable email sending setup that makes him public enemy
#1 as far as gmail is concerned. So most of his emails seem to be
trapped by gmail as spam, resulting in me not seeing them. - Linus ]
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (49 commits)
crypto: s390 - Fix aes-cbc IV corruption
crypto: omap-aes - Fix CTR mode counter length
crypto: omap-sham - Add missing modalias
padata: make the sequence counter an atomic_t
crypto: caam - Modify the interface layers to use JR API's
crypto: caam - Add API's to allocate/free Job Rings
crypto: caam - Add Platform driver for Job Ring
hwrng: msm - Add PRNG support for MSM SoC's
ARM: DT: msm: Add Qualcomm's PRNG driver binding document
crypto: skcipher - Use eseqiv even on UP machines
crypto: talitos - Simplify key parsing
crypto: picoxcell - Simplify and harden key parsing
crypto: ixp4xx - Simplify and harden key parsing
crypto: authencesn - Simplify key parsing
crypto: authenc - Export key parsing helper function
crypto: mv_cesa: remove deprecated IRQF_DISABLED
hwrng: OMAP3 ROM Random Number Generator support
crypto: sha256_ssse3 - also test for BMI2
crypto: mv_cesa - Remove redundant of_match_ptr
crypto: sahara - Remove redundant of_match_ptr
...
Use the common helper function crypto_authenc_extractkeys() for key
parsing.
Cc: Kim Phillips <kim.phillips@freescale.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Mathias Krause <mathias.krause@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Powerpc is a mess of implicit includes by prom.h. Add the necessary
explicit includes to drivers in preparation of prom.h cleanup.
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Grant Likely <grant.likely@linaro.org>
Use sg_pcopy_to_buffer() which is better than the function previously used.
Because it doesn't do kmap/kunmap for skipped pages.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: Douglas Gilbert <dgilbert@interlog.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Horia Geanta <horia.geanta@freescale.com>
Cc: Imre Deak <imre.deak@intel.com>
Cc: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This reverts commit e763eb699b.
Current IPsec ESN implementation for authencesn(cbc(aes), hmac(sha))
(separate encryption and integrity algorithms) does not conform
to RFC4303.
ICV is generated by hashing the sequence
SPI, SeqNum-High, SeqNum-Low, IV, Payload
instead of
SPI, SeqNum-Low, IV, Payload, SeqNum-High.
Cc: <stable@vger.kernel.org> # 3.8, 3.7
Reported-by: Chaoxing Lin <Chaoxing.Lin@ultra-3eti.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Reviewed-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Convert cpu_to_be16(be16_to_cpu(E1) + E2) to use be16_add_cpu().
dpatch engine is used to auto generate this patch.
(https://github.com/weiyj/dpatch)
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Support for ESNs (extended sequence numbers).
Tested with strongswan on a P2020RDB back-to-back setup.
Extracted from /etc/ipsec.conf:
esp=aes-sha1-esn-modp4096!
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Generate a link table in case assoc data is a scatterlist.
While at it, add support for handling non-contiguous assoc data and iv.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It's more natural to think of these vars as bool rather than int.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
talitos_edesc_alloc does not need hash_result param.
Checking whether dst scatterlist is NULL or not is all that is required.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
For IPsec encryption, in the case when:
-the input buffer is fragmented (edesc->src_nents > 0)
-the output buffer is not fragmented (edesc->dst_nents = 0)
the ICV is not output in the link table, but after the encrypted payload.
Copying the ICV must be avoided in this case; consequently the condition
edesc->dma_len > 0 must be more specific, i.e. must depend on the type
of the output buffer - fragmented or not.
Testing was performed by modifying testmgr to support src != dst,
since currently native kernel IPsec does in-place encryption
(src == dst).
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
the entry points and geniv definitions for all aead,
ablkcipher, and hash algorithms are all common; move them to a
single assignment in talitos_alg_alloc().
This assumes it's ok to assign a setkey() on non-hmac algs.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
lighten driver_algs[] by moving them to talitos_alg_alloc().
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With this, now all combinations of
CBC: AES, 3DES-EDE
with
HMAC: SHA-1, SHA-224, SHA-256, SHA-384, SHA-512
are supported.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch exports the talitos_submit function so that on
need basis same can be used by other entities.
Signed-off-by: Sandeep Malik <Sandeep.Malik@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the talitos structure definitions from c file to its
header file so that the same can be shared on need basis.
Signed-off-by: Sandeep Malik <Sandeep.Malik@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Access to global talitos registers must be protected for the case when
affinities are configured such that primary and secondary talitos irqs
run on different cpus.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The added CRYPTO_ALG_KERN_DRIVER_ONLY indicates whether a cipher
is only available via a kernel driver. If the cipher implementation
might be available by using an instruction set or by porting the
kernel code, then it must not be set.
Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC2/3 h/w doesn't have a dedicated interrupt for errors,
and the only callsite for talitos_error has already done
the type conversion, so simplify talitos_error to take a
pointer to a struct device.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As prescribed by Linus:
https://lkml.org/lkml/2011/12/2/290
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fix a kfree to an invalid address which causes an oops when running
on SEC v2.0 h/w (introduced in commit 702331b "crypto: talitos - add
hmac algorithms").
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts the drivers in drivers/crypto/* to use the
module_platform_driver() macro which makes the code smaller and a bit
simpler.
Cc: James Hsiao <jhsiao@amcc.com>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Cc: Kim Phillips <kim.phillips@freescale.com>
Cc: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Axel Lin <axel.lin@gmail.com>
Acked-by: Vladimir Zapolskiy <vzapolskiy@gmail.com>
Acked-by: Jamie Iles <jamie@jamieiles.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some later SEC v3.x are equipped with a second IRQ line.
By correctly assigning IRQ affinity, this feature can be
used to increase performance on dual core parts, like the
MPC8572E and P2020E.
The existence of the 2nd IRQ is determined from the device
node's interrupt property. If present, the driver remaps
two of four channels, which in turn makes those channels
trigger their interrupts on the 2nd line instead of the first.
To handle single- and dual-IRQ combinations efficiently,
talitos gets two new interrupt handlers and back-half workers.
[includes a fix to MCR_LO's address.]
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add a reg member to the channel struct and use it to
access channels.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
talitos prints every algorithm it registers at module load time.
Algorithms are being added that make for an excessively noisy console
(latest HMACs patch makes an SEC 3.1 print 20 lines).
Instead, display the SEC h/w version number, and inform the
user of algorithm registration status in /proc/crypto, like so:
talitos ffe30000.crypto: fsl,sec3.1 algorithms registered in /proc/crypto
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add these hmac algorithms to talitos:
hmac(md5),
hmac(sha1),
hmac(sha224),
hmac(sha256),
hmac(sha384),
hmac(sha512).
These are all type ahash.
Signed-off-by: Lee Nipper <lee.nipper@gmail.com>
Fixed up to not register HMAC algorithms on sec2.0 devices.
Rationale (from Lee):
on an 8349E Rev1.1, there's a problem with hmac for any talitos
hmac sequence requiring an intermediate hash context (Pointer
DWORD 1); the result is an incorrect hmac. An intermediate hash
context is required for something longer than (65536-blocksize),
and for other cases when update/finup/final are used inefficiently.
Interestingly, a normal hash (without hmac) works perfectly
when using an intermediate context.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The CDPR (Current Descriptor Pointer Register) can be unreliable
when trying to locate an offending descriptor. Handle that case by
(a) not OOPSing, and (b) reverting to the machine internal copy of
the descriptor header in order to report the correct execution unit
error.
Note: printing all execution units' ISRs is not effective because it
results in an internal time out (ITO) error and the EU resetting its
ISR value (at least when specifying an invalid key length on an SEC
2.2/MPC8313E).
Reported-by: Sven Schnelle <svens@stackframe.org>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto/ablkcipher.c's setkey() has already checked against the min, max
key sizes before it calls here, and all max_keysize assignments in the
algorithm template array do not exceed TALITOS_MAX_KEY_SIZE.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
