Commit graph

53 commits

Author SHA1 Message Date
Jakub Kicinski
be093a80df tools: ynl-gen: inherit policy in multi-attr
Instead of reimplementing policies in MutliAttr for every
underlying type forward the calls to the base type.
This will be needed for DPLL which uses a multi-attr nest,
and currently gets an invalid NLA_NEST policy generated.

Reviewed-by: Jacob Keller <Jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-13 11:46:46 -07:00
Jakub Kicinski
10c4d2a7b8 tools: ynl-gen: correct enum policies
Scalar range validation assumes enums start at 0.
Teach it to properly calculate the value range.

Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-13 11:46:46 -07:00
Jakub Kicinski
2c9d47a095 tools: ynl-gen: resolve enum vs struct name conflicts
Ethtool has an attribute set called stringset, from which
we'll generate struct ethtool_stringset. Unfortunately,
the old ethtool header declares enum ethtool_stringset
(the same name), to which compilers object.

This seems unavoidable. Check struct names against known
constants and append an underscore if conflict is detected.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-12 11:01:03 +01:00
Jakub Kicinski
dddc9f53da tools: ynl-gen: don't generate enum types if unnamed
If attr set or enum has empty enum name we need to use u32 or int
as function arguments and struct members.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-12 11:01:03 +01:00
Jakub Kicinski
ed2042cc77 netlink: specs: support setting prefix-name per attribute
Ethtool's PSE PoDL has a attr nest with different prefixes:

/* Power Sourcing Equipment */
enum {
	ETHTOOL_A_PSE_UNSPEC,
	ETHTOOL_A_PSE_HEADER,			/* nest - _A_HEADER_* */
	ETHTOOL_A_PODL_PSE_ADMIN_STATE,		/* u32 */
	ETHTOOL_A_PODL_PSE_ADMIN_CONTROL,	/* u32 */
	ETHTOOL_A_PODL_PSE_PW_D_STATUS,		/* u32 */

Header has a prefix of ETHTOOL_A_PSE_ and other attrs prefix of
ETHTOOL_A_PODL_PSE_ we can't cover them uniformly.
If PODL was after PSE life would be easy.

Now we either need to add prefixes to attr names which is yucky
or support setting prefix name per attr.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-12 11:01:02 +01:00
Jakub Kicinski
33eedb0071 tools: ynl-gen: record extra args for regen
ynl-regen needs to know the arguments used to generate a file.
Record excluded ops and, while at it, user headers.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-12 11:01:02 +01:00
Jakub Kicinski
008bcd6835 tools: ynl-gen: support excluding tricky ops
The ethtool family has a small handful of quite tricky ops
and a lot of simple very useful ops. Teach ynl-gen to skip
ops so that we can bypass the tricky ones.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-12 11:01:02 +01:00
Jakub Kicinski
76abff37f0 tools: ynl-gen: support / skip pads on the way to kernel
Kernel does not have padding requirements for 64b attrs.
We can ignore pad attrs.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
6f96ec73cb tools: ynl-gen: don't pass op_name to RenderInfo
The op_name argument is barely used and identical to op.name
in all cases.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
6da3424fd6 tools: ynl-gen: support code gen for events
Netlink specs support both events and notifications (former can
define their own message contents). Plug in missing code to
generate types, parsers and include events into notification
tables.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
ced1568862 tools: ynl-gen: sanitize notification tracking
Don't modify the raw dicts (as loaded from YAML) to pretend
that the notify attributes also exist on the ops. This makes
the code easier to follow.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
f2ba1e5e22 tools: ynl-gen: stop generating common notification handlers
Common notification handler was supposed to be a way for the user
to parse the notifications from a socket synchronously.
I don't think we'll end up using it, ynl_ntf_check() works for
all known use cases.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
e4ea3cc684 tools: ynl-gen: get attr type outside of if()
Reading attr type with mnl_attr_get_type() for each condition
leads to most conditions being longer than 80 chars.
Avoid this by reading the type to a variable on the stack.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
2c0f146686 tools: ynl-gen: combine else with closing bracket
Code gen currently prints:

  }
  else if (...

This is really ugly. Fix it by delaying printing of closing
brackets in anticipation of else coming along.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
820343ccbb tools: ynl-gen: complete the C keyword list
C keywords need to be avoided when naming things.
Complete the list (ethtool has at least one thing called "auto").

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
30b5c720e1 tools: ynl-gen: cleanup user space header includes
Bots started screaming that we're including stdlib.h twice.
While at it move string.h into a common spot and drop stdio.h
which we don't need.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5464
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5466
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5467
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
168dea20ec tools: ynl-gen: don't generate forward declarations for policies
Now that all nested types have structs and are sorted topologically
there should be no need to generate forward declarations for policies.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
eae7af21bd tools: ynl-gen: walk nested types in depth
So far we had only created structures for nested types nested
directly in messages (second level of attrs so to speak).
Walk types in depth to support deeper nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
37487f93b1 tools: ynl-gen: inherit struct use info
We only render parse and netlink generation helpers as needed,
to avoid generating dead code. Propagate the information from
first- and second-layer attribute sets onto all children.
Otherwise devlink won't work, it has a lot more levels of nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6afaa0ef9b tools: ynl-gen: try to sort the types more intelligently
We need to sort the structures to avoid the need for forward
declarations. While at it remove the sort of structs when
rendering, it doesn't do anything.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
ff6db4b58c tools: ynl-gen: enable code gen for directional specs
I think that user space code gen for directional specs
works after recent changes. Let them through.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6f115d4575 tools: ynl-gen: refactor strmap helper generation
Move generating strmap lookup function to a helper.
No functional changes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
9858bfc271 tools: ynl-gen: use enum names in op strmap more carefully
In preparation for supporting families which use different msg
ids to and from the kernel - make sure the ids in op strmap
are correct. The map is expected to be used mostly for notifications,
don't generate a separate map for the "to kernel" direction.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
58da455b31 tools: ynl-gen: improve unwind on parsing errors
When parsing multi-attr we count the objects and then allocate
an array to hold the parsed objects. If an attr space has multiple
multi-attr objects, however, if parsing the first array fails
we'll leave the object count for the second even tho the second
array was never allocated.

This may cause crashes when freeing objects on error.

Count attributes to a variable on the stack and only set the count
in the object once the memory was allocated.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
2cc9671a82 tools: ynl-gen: fill in support for MultiAttr scalars
The handshake family needs support for MultiAttr scalars.
Right now we only support code gen for MultiAttr nested
types.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
a99bfdf647 tools: ynl-gen: clean up stray new lines at the end of reply-less requests
Do not print empty lines before closing brackets.

Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:31 -07:00
Jakub Kicinski
59d814f0f2 tools: ynl-gen: generate static descriptions of notifications
Notifications may come in at any time. The family must be always
ready to parse a random incoming notification. Generate notification
table for parsing and tell YNL which request we're processing
to distinguish responses from notifications.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
8cb6afb335 tools: ynl-gen: switch to family struct
We'll want to store static info about the family soon.
Generate a struct. This changes creation from, e.g.:

	 ys = ynl_sock_create("netdev", &yerr);
to:
	 ys = ynl_sock_create(&ynl_netdev_family, &yerr);

on user's side.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5d58f911c7 tools: ynl-gen: generate alloc and free helpers for req
We expect user to allocate requests with calloc(),
make things a bit more consistent and provide helpers.
Generate free calls, too.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
dc0956c98f tools: ynl-gen: move the response reading logic into YNL
We generate send() and recv() calls and all msg handling for
each operation. It's a lot of repeated code and will only grow
with notification handling. Call back to a helper YNL lib instead.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
21b6e30278 tools: ynl-gen: generate enum-to-string helpers
It's sometimes useful to print the name of an enum value,
flag or name of the op. Python can do it, add C helper
code gen for getting names of things.

Example:

  static const char * const netdev_xdp_act_strmap[] = {
	[0] = "basic",
	[1] = "redirect",
	[2] = "ndo-xmit",
	[3] = "xsk-zerocopy",
	[4] = "hw-offload",
	[5] = "rx-sg",
	[6] = "ndo-xmit-sg",
  };

  const char *netdev_xdp_act_str(enum netdev_xdp_act value)
  {
	value = ffs(value) - 1;
	if (value < 0 || value >= (int)MNL_ARRAY_SIZE(netdev_xdp_act_strmap))
		return NULL;
	return netdev_xdp_act_strmap[value];
  }

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
eef9b794ea tools: ynl-gen: add error checking for nested structs
Parsing nested types may return an error, propagate it.
Not marking as a fix, because nothing uses YNL upstream.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5605f10237 tools: ynl-gen: loosen type consistency check for events
Both event and notify types are always consistent. Rewrite
the condition checking if we can reuse reply types to be
less picky and let notify thru.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
67c65ce762 tools: ynl-gen: don't override pure nested struct
For pure structs (parsed nested attributes) we track what
forms of the struct exist in request and reply directions.
Make sure we don't overwrite the recorded struct each time,
otherwise the information is lost.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
6ad49839ba tools: ynl-gen: fix unused / pad attribute handling
Unused and Pad attributes don't carry information.
Unused should never exist, and be rejected.
Pad should be silently skipped.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
91dfaef243 tools: ynl-gen: add extra headers for user space
Make sure all relevant headers are included, we allocate memory,
use memcpy() and Linux types without including the headers.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
9b66ee06e5 net: ynl: prefix uAPI header include with uapi/
To keep things simple we used to include the uAPI header
in the kernel in the #include <linux/$family.h> format.
This works well enough, most of the genl families should
have headers in include/net/ so linux/$family.h ends up
referring to the uAPI header, anyway. And if it doesn't
no big deal, we'll just include more info than we need.

Unless that is there is a naming conflict. Someone recently
created include/linux/psp.h which will be a problem when
supporting the PSP protocol. (I'm talking about
work-in-progress patches, but it's just a proof that assuming
lack of name conflicts was overly optimistic.)

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-05-26 10:30:14 +01:00
Jakub Kicinski
4c6170d1ae tools: ynl: default to treating enums as flags for mask generation
I was a bit too optimistic in commit bf51d27704 ("tools: ynl: fix
get_mask utility routine"), not every mask we use is necessarily
coming from an enum of type "flags". We also allow flipping an
enum into flags on per-attribute basis. That's done by
the 'enum-as-flags' property of an attribute.

Restore this functionality, it's not currently used by any in-tree
family.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-03-27 08:56:04 +01:00
Jakub Kicinski
56c874f7db tools: ynl: skip the explicit op array size when not needed
Jiri suggests it reads more naturally to skip the explicit
array size when possible. When we export the symbol we want
to make sure that the size is right but for statics its
not needed.

Link: https://lore.kernel.org/r/20230321044159.1031040-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-21 21:45:31 -07:00
Jakub Kicinski
cfab77c0b5 ynl: make the tooling check the license
The (only recently documented) expectation is that all specs
are under a certain license, but we don't actually enforce it.
What's worse we then go ahead and assume the license was right,
outputting the expected license into generated files.

Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-16 21:22:44 -07:00
Jakub Kicinski
4e16b6a748 ynl: broaden the license even more
I relicensed Netlink spec code to GPL-2.0 OR BSD-3-Clause but
we still put a slightly different license on the uAPI header
than the rest of the code. Use the Linux-syscall-note on all
the specs and all generated code. It's moot for kernel code,
but should not hurt. This way the licenses match everywhere.

Cc: Chuck Lever <chuck.lever@oracle.com>
Fixes: 37d9df224d ("ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause")
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-16 21:20:32 -07:00
Lorenzo Bianconi
8f76a4f80f tools: ynl: fix render-max for flags definition
Properly manage render-max property for flags definition type
introducing mask value and setting it to (last_element << 1) - 1
instead of adding max value set to last_element + 1

Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-10 21:33:47 -08:00
Jakub Kicinski
6517a60b03 tools: ynl: move the enum classes to shared code
Move bulk of the EnumSet and EnumEntry code to shared
code for reuse by cli.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-08 23:28:21 -08:00
Jakub Kicinski
37d9df224d ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause
I was intending to make all the Netlink Spec code BSD-3-Clause
to ease the adoption but it appears that:
 - I fumbled the uAPI and used "GPL WITH uAPI note" there
 - it gives people pause as they expect GPL in the kernel
As suggested by Chuck re-license under dual. This gives us benefit
of full BSD freedom while fulfilling the broad "kernel is under GPL"
expectations.

Link: https://lore.kernel.org/all/20230304120108.05dd44c5@kernel.org/
Link: https://lore.kernel.org/r/20230306200457.3903854-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-03-07 13:44:30 -08:00
Jakub Kicinski
ad4fafcde5 tools: ynl: use 1 as the default for first entry in attrs/ops
Pretty much all families use value: 1 or reserve as unspec
the first entry in attribute set and the first operation.
Make this the default. Update documentation (the doc for
values of operations just refers back to doc for attrs
so updating only attrs).

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-03-03 08:22:39 +00:00
Jakub Kicinski
f7cf644796 tools: ynl-gen: fix single attribute structs with attr 0 only
Chuck run into an issue with a single-element attr-set which
only has an attr with value of 0. The search for max attr in
a struct records attrs with value larger than 0 only (max_val
is set to 0 at the start). Adjust the comparison, alternatively
max_val could be init'ed to -1. Somehow picking the last attr
of a value seems like a good idea in general.

Reported-by: Chuck Lever III <chuck.lever@oracle.com>
Fixes: be5bea1cc0 ("net: add basic C code generators for Netlink")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-02-24 11:55:47 -08:00
Jakub Kicinski
981cbcb030 tools: net: use python3 explicitly
The scripts require Python 3 and some distros are dropping
Python 2 support.

Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
30a5c6c810 tools: ynl: use the common YAML loading and validation code
Adapt the common object hierarchy in code gen and CLI.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
eaf317e7d2 tools: ynl-gen: prevent do / dump reordering
An earlier fix tried to address generated code jumping around
one code-gen run to another. Turns out dict()s are already
ordered since Python 3.7, the problem is that we iterate over
operation modes using a set(). Sets are unordered in Python.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-31 20:36:03 -08:00
Jakub Kicinski
3a43ded081 tools: ynl: store ops in ordered dict to avoid random ordering
When rendering code we should walk the ops in the order in which
they are declared in the spec. This is both more intuitive and
prevents code from jumping around when hashing in the dict changes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-01-26 16:32:41 -08:00