We avoid printing forward declarations and prototypes for most
types by sorting things topologically. But if structs nest we
do need the forward declarations, there's no other way.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-9-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
To avoid infinite nesting store recursive structs by pointer.
If recursive struct is placed in the op directly - the first
instance can be stored by value. That makes the code much
less of a pain for majority of practical uses.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-8-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We try to keep the structures and helpers "topologically sorted",
to avoid forward declarations. When recursive nests are at play
we need to sort twice, because structs which end up being marked
as recursive will get a full set of forward declarations, so we
should ignore them for the purpose of sorting.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Track which nests are recursive. Non-recursive nesting gets
rendered in C as directly nested structs. For recursive
ones we need to put a pointer in, rather than full struct.
Track this information, no change to generated code, yet.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Fill in more empty handlers for TypeUnused. When 'unused'
attr gets specified in a nested set we have to cleanly
skip it during code generation.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Support genetlink families using simple fixed headers.
Assume fixed header is identical for all ops of the family for now.
Fixed headers are added to the request and reply structs as a _hdr
member, and copied to/from netlink messages appropriately.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 30c9020015 ("tools: ynl-gen: use enum name from the spec")
added pre-cooked user type for enums. Use it to fix ignoring
enum-name provided in the spec.
This changes a type in struct ethtool_tunnel_udp_entry but is
generally inconsequential for current families.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The code gen generates a prototype for dump request free
in the header, but no implementation in the source.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20231213231432.2944749-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 2b7ac0c87d ("tools: ynl-gen: don't touch the output file if
content is the same") is working too well. It was added so that
ynl-regen -f doesn't make us rebuild half of the kernel, if there
are no actual changes in any generated code.
When ynl-gen-c is called by make, however, we're better off trusting
make's tracking and overwrite the file. Otherwise if output is identical
we won't update file timestamps and make will retry code gen on every
invocation.
Link: https://lore.kernel.org/r/20231129193622.2912353-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
struct ynl_req_state carries reply-related info from generated code
into generic YNL code. While we don't need reply info to execute
a request without a reply, we still need to pass in the struct, because
it's also where we get the pointer to struct ynl_sock from. Passing NULL
results in crashes if kernel returns an error or an unexpected reply.
Fixes: dc0956c98f ("tools: ynl-gen: move the response reading logic into YNL")
Link: https://lore.kernel.org/r/20231126225858.2144136-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The enum name used for id-to-str table does not handle
the enum-name override in the spec correctly.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
If a new family is ever added with a dash in the name
the C codegen will break. Make sure we use the "safe"
form of the name consistently.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
32bit builds generate the following warning when we use a u32-max
in range validation:
warning: decimal constant 4294967295 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
The range values are u64, slap ULL/LL on all of them just
to avoid such noise.
There's currently no code using full range validation, but
it will matter in the upcoming page-pool introspection.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
We don't support CRUD-inspired message types in YNL too well.
One aspect that currently trips us up is the fact that single
message ID can be used in multiple commands (as the response).
This leads to duplicate entries in the id-to-string tables:
devlink-user.c:19:34: warning: initialized field overwritten [-Woverride-init]
19 | [DEVLINK_CMD_PORT_NEW] = "port-new",
| ^~~~~~~~~~
devlink-user.c:19:34: note: (near initialization for ‘devlink_op_strmap[7]’)
Fixes tag points at where the code was generated, the "real" problem
is that the code generator does not support CRUD.
Fixes: f2f9dd164d ("netlink: specs: devlink: add the remaining command to generate complete split_ops")
Link: https://lore.kernel.org/r/20231123030558.1611831-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
I often regenerate all YNL files in the tree to make sure they
are in sync with the codegen and specs. Generator rewrites
the files unconditionally, so since make looks at file modification
time to decide what to rebuild - my next build takes longer.
We already generate the code to a tempfile most of the time,
only overwrite the target when we have to.
Before:
$ stat include/uapi/linux/netdev.h
File: include/uapi/linux/netdev.h
Size: 2307 Blocks: 8 IO Block: 4096 regular file
Access: 2023-10-27 15:19:56.347071940 -0700
Modify: 2023-10-27 15:19:45.089000900 -0700
Change: 2023-10-27 15:19:45.089000900 -0700
Birth: 2023-10-27 15:19:45.088000894 -0700
$ ./tools/net/ynl/ynl-regen.sh -f
[...]
$ stat include/uapi/linux/netdev.h
File: include/uapi/linux/netdev.h
Size: 2307 Blocks: 8 IO Block: 4096 regular file
Access: 2023-10-27 15:19:56.347071940 -0700
Modify: 2023-10-27 15:22:18.417968446 -0700
Change: 2023-10-27 15:22:18.417968446 -0700
Birth: 2023-10-27 15:19:45.088000894 -0700
After:
$ stat include/uapi/linux/netdev.h
File: include/uapi/linux/netdev.h
Size: 2307 Blocks: 8 IO Block: 4096 regular file
Access: 2023-10-27 15:22:41.520114221 -0700
Modify: 2023-10-27 15:22:18.417968446 -0700
Change: 2023-10-27 15:22:18.417968446 -0700
Birth: 2023-10-27 15:19:45.088000894 -0700
$ ./tools/net/ynl/ynl-regen.sh -f
[...]
$ stat include/uapi/linux/netdev.h
File: include/uapi/linux/netdev.h
Size: 2307 Blocks: 8 IO Block: 4096 regular file
Access: 2023-10-27 15:22:41.520114221 -0700
Modify: 2023-10-27 15:22:18.417968446 -0700
Change: 2023-10-27 15:22:18.417968446 -0700
Birth: 2023-10-27 15:19:45.088000894 -0700
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20231027223408.1865704-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Page pool code is compiled conditionally, but the operations
are part of the shared netlink family. We can handle this
by reporting empty list of pools or -EOPNOTSUPP / -ENOSYS
but the cleanest way seems to be removing the ops completely
at compilation time. That way user can see that the page
pool ops are not present using genetlink introspection.
Same way they'd check if the kernel is "new enough" to
support the ops.
Extend the specs with the ability to specify the config
condition under which op (and its policies, etc.) should
be hidden.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20231025162253.133159-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
struct nla_policy is usually constant itself, but unless
we make the ranges inside constant we won't be able to
make range structs const. The ranges are not modified
by the core.
Reviewed-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: David Ahern <dsahern@kernel.org>
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20231025162204.132528-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Jiri Pirko says:
====================
devlink: finish conversion to generated split_ops
This patchset converts the remaining genetlink commands to generated
split_ops and removes the existing small_ops arrays entirely
alongside with shared netlink attribute policy.
Patches #1-#6 are just small preparations and small fixes on multiple
places. Note that couple of patches contain the "Fixes"
tag but no need to put them into -net tree.
Patch #7 is a simple rename preparation
Patch #8 is the main one in this set and adds actual definitions of cmds
in to yaml file.
Patches #9-#10 finalize the change removing bits that are no longer in
use.
====================
Link: https://lore.kernel.org/r/20231021112711.660606-1-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Due to the check in RenderInfo class constructor, type_consistent
flag is set to False to avoid rendering the same response parsing
helper for do and dump ops. However, in case there is no do, the helper
needs to be rendered for dump op. So split check to achieve that.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://lore.kernel.org/r/20231021112711.660606-4-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Introduce support for attribute type bitfield32.
Note that since the generated code works with struct nla_bitfield32,
the generator adds netlink.h to the list of includes for userspace
headers in case any bitfield32 is present.
Note that this is added only to genetlink-legacy scheme as requested
by Jakub Kicinski.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://lore.kernel.org/r/20231021112711.660606-3-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
checkpatch gets confused and treats __attribute__ as a function call.
It complains about white space before "(":
WARNING:SPACING: space prohibited between function name and open parenthesis '('
+ struct netdev_queue_get_rsp obj __attribute__ ((aligned (8)));
No spaces wins in the kernel:
$ git grep 'attribute__((.*aligned(' | wc -l
480
$ git grep 'attribute__ ((.*aligned (' | wc -l
110
$ git grep 'attribute__ ((.*aligned(' | wc -l
94
$ git grep 'attribute__((.*aligned (' | wc -l
63
So, whatever, change the codegen.
Note that checkpatch also thinks we should use __aligned(),
but this is user space code.
Link: https://lore.kernel.org/all/202310190900.9Dzgkbev-lkp@intel.com/
Acked-by: Stanislav Fomichev <sdf@google.com>
Reviewed-by: Amritha Nambiar <amritha.nambiar@intel.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20231020221827.3436697-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Support uint / sint types in specs and YNL.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
uint/sint support will add more logic to mnl_type(),
deduplicate it and make it more accessible.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Extend the support to full range of min/max checks.
None of the existing YNL families required complex integer validation.
The support is less than trivial, because we try to keep struct nla_policy
tiny the min/max members it holds in place are s16. Meaning we can only
express checks in range of s16. For larger ranges we need to define
a structure and link it in the policy.
Link: https://lore.kernel.org/r/20231018163917.2514503-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
For range validation we'll need to know if any individual
attribute is used on input (i.e. whether we will generate
a policy for it). Track this information.
Link: https://lore.kernel.org/r/20231018163917.2514503-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The code supports dumps with no input attributes currently
thru a combination of special-casing and luck.
Clean up the handling of ops with no inputs. Create empty
Structs, and skip printing of empty types.
This makes dos with no inputs work.
Tested-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Link: https://lore.kernel.org/r/20231006135032.3328523-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Chuck points out that we should use the uapi-header property
when generating the guard. Otherwise we may generate the same
guard as another file in the tree.
Tested-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
We use a tempfile for code generation, to avoid wiping the target
file out if the code generator crashes. File contents are copied
from tempfile to actual destination at the end of main().
uAPI generation is relatively simple so when generating the uAPI
header we return from main() early, and never reach the "copy code
over" stage. Since commit under Fixes uAPI headers are not updated
by ynl-gen.
Move the copy/commit of the code into CodeWriter, to make it
easier to call at any point in time. Hook it into the destructor
to make sure we don't miss calling it.
Fixes: f65f305ae0 ("tools: ynl-gen: use temporary file for rendering")
Link: https://lore.kernel.org/r/20230824212431.1683612-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Differentiate between empty list and None for member lists.
New families may want to create request responses with no attribute.
If we treat those the same as None we end up rendering
a full parsing policy in user space, instead of an empty one.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230824003056.1436637-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We look for attributes inside do.request, but there's another
layer of nesting in the spec, look inside do.request.attributes.
This bug had no effect as all global policies we generate (fou)
seem to be full, anyway, and we treat full and empty the same.
Next patch will change the treatment of empty policies.
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230824003056.1436637-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Currently any error during render leads to output an empty file.
That is quite annoying when using tools/net/ynl/ynl-regen.sh
which git greps files with content of "YNL-GEN.." and therefore ignores
empty files. So once you fail to regen, you have to checkout the file.
Avoid that by rendering to a temporary file first, only at the end
copy the content to the actual destination.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
When dont-validate flags are filtered out for do/dump op, the list may
be empty. In that case, avoid rendering the validate field.
Fixes: fa8ba3502a ("ynl-gen-c.py: render netlink policies static for split ops")
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230808090344.1368874-1-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When policies are rendered for split ops, they are consumed in the same
file. No need to expose them for user outside, make them static.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230803111340.1074067-5-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Directional model limitation is only applicable for uapi mode.
For kernel mode, the code is generated correctly using right cmd values
for do/dump requests. Lift the limitation.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230803111340.1074067-4-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
For split ops, do and dump has different meaningful values in
validate field.
Fix the rendering to allow the values per op type as follows:
do: strict
dump: dump, strict-dump
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230803111340.1074067-3-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Simon mentioned in another thread that it makes kdoc happy
and Jakub confirms that commit e27cb89a22 ("scripts: kernel-doc: support
private / public marking for enums") actually added the needed
support.
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/r/20230727163001.3952878-3-sdf@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of reimplementing policies in MutliAttr for every
underlying type forward the calls to the base type.
This will be needed for DPLL which uses a multi-attr nest,
and currently gets an invalid NLA_NEST policy generated.
Reviewed-by: Jacob Keller <Jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Scalar range validation assumes enums start at 0.
Teach it to properly calculate the value range.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Ethtool has an attribute set called stringset, from which
we'll generate struct ethtool_stringset. Unfortunately,
the old ethtool header declares enum ethtool_stringset
(the same name), to which compilers object.
This seems unavoidable. Check struct names against known
constants and append an underscore if conflict is detected.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
If attr set or enum has empty enum name we need to use u32 or int
as function arguments and struct members.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Ethtool's PSE PoDL has a attr nest with different prefixes:
/* Power Sourcing Equipment */
enum {
ETHTOOL_A_PSE_UNSPEC,
ETHTOOL_A_PSE_HEADER, /* nest - _A_HEADER_* */
ETHTOOL_A_PODL_PSE_ADMIN_STATE, /* u32 */
ETHTOOL_A_PODL_PSE_ADMIN_CONTROL, /* u32 */
ETHTOOL_A_PODL_PSE_PW_D_STATUS, /* u32 */
Header has a prefix of ETHTOOL_A_PSE_ and other attrs prefix of
ETHTOOL_A_PODL_PSE_ we can't cover them uniformly.
If PODL was after PSE life would be easy.
Now we either need to add prefixes to attr names which is yucky
or support setting prefix name per attr.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
ynl-regen needs to know the arguments used to generate a file.
Record excluded ops and, while at it, user headers.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The ethtool family has a small handful of quite tricky ops
and a lot of simple very useful ops. Teach ynl-gen to skip
ops so that we can bypass the tricky ones.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
