Commit graph

94 commits

Author SHA1 Message Date
Jakub Kicinski
76abff37f0 tools: ynl-gen: support / skip pads on the way to kernel
Kernel does not have padding requirements for 64b attrs.
We can ignore pad attrs.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
6f96ec73cb tools: ynl-gen: don't pass op_name to RenderInfo
The op_name argument is barely used and identical to op.name
in all cases.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
6da3424fd6 tools: ynl-gen: support code gen for events
Netlink specs support both events and notifications (former can
define their own message contents). Plug in missing code to
generate types, parsers and include events into notification
tables.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
ced1568862 tools: ynl-gen: sanitize notification tracking
Don't modify the raw dicts (as loaded from YAML) to pretend
that the notify attributes also exist on the ops. This makes
the code easier to follow.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
d0915d64c3 tools: ynl: regen: stop generating common notification handlers
Remove unused notification handlers.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
f2ba1e5e22 tools: ynl-gen: stop generating common notification handlers
Common notification handler was supposed to be a way for the user
to parse the notifications from a socket synchronously.
I don't think we'll end up using it, ynl_ntf_check() works for
all known use cases.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
7234415b8f tools: ynl: regen: regenerate the if ladders
Renegate the code to combine } and else and use tmp variable
to store type.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
e4ea3cc684 tools: ynl-gen: get attr type outside of if()
Reading attr type with mnl_attr_get_type() for each condition
leads to most conditions being longer than 80 chars.
Avoid this by reading the type to a variable on the stack.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
2c0f146686 tools: ynl-gen: combine else with closing bracket
Code gen currently prints:

  }
  else if (...

This is really ugly. Fix it by delaying printing of closing
brackets in anticipation of else coming along.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
820343ccbb tools: ynl-gen: complete the C keyword list
C keywords need to be avoided when naming things.
Complete the list (ethtool has at least one thing called "auto").

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
9b52fd4b63 tools: ynl: regen: cleanup user space header includes
Remove unnecessary includes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
30b5c720e1 tools: ynl-gen: cleanup user space header includes
Bots started screaming that we're including stdlib.h twice.
While at it move string.h into a common spot and drop stdio.h
which we don't need.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5464
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5466
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5467
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 14:40:31 -07:00
Jakub Kicinski
7ec5d48fdb Revert "tools: ynl: Remove duplicated include in handshake-user.c"
This reverts commit e7c5433c5a.

Commit e7c5433c5a ("tools: ynl: Remove duplicated include
in handshake-user.c") was applied too hastily. It changes
an auto-generated file, and there's already a proper fix
on the list.

Link: https://lore.kernel.org/all/ZIMPLYi%2FxRih+DlC@nanopsycho/
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-09 11:01:04 -07:00
Yang Li
e7c5433c5a tools: ynl: Remove duplicated include in handshake-user.c
./tools/net/ynl/generated/handshake-user.c: stdlib.h is included more than once.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5464
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-06-09 11:36:41 +01:00
Jakub Kicinski
fff8660b54 tools: ynl: add sample for devlink
Add a sample to show off how to issue basic devlink requests.
For added testing issue get requests while walking a dump.

$ ./devlink
netdevsim/netdevsim1:
    driver: netdevsim
    running fw:
        fw.mgmt: 10.20.30
    ...
netdevsim/netdevsim2:
    driver: netdevsim
    running fw:
        fw.mgmt: 10.20.30
    ...

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
5d1a30eb98 tools: ynl: generate code for the devlink family
Admittedly the devlink.yaml spec is fairly limitted,
it only covers basic device get and info-get ops.
That's sufficient to be useful (monitoring FW versions
in the fleet). Plus it gives us a chance to exercise
deep nesting and directional messaging in YNL.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
0a94712196 tools: ynl-gen: don't generate forward declarations for policies - regen
Renegerate code after dropping forward declarations for policies.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
168dea20ec tools: ynl-gen: don't generate forward declarations for policies
Now that all nested types have structs and are sorted topologically
there should be no need to generate forward declarations for policies.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
eae7af21bd tools: ynl-gen: walk nested types in depth
So far we had only created structures for nested types nested
directly in messages (second level of attrs so to speak).
Walk types in depth to support deeper nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
37487f93b1 tools: ynl-gen: inherit struct use info
We only render parse and netlink generation helpers as needed,
to avoid generating dead code. Propagate the information from
first- and second-layer attribute sets onto all children.
Otherwise devlink won't work, it has a lot more levels of nesting.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6afaa0ef9b tools: ynl-gen: try to sort the types more intelligently
We need to sort the structures to avoid the need for forward
declarations. While at it remove the sort of structs when
rendering, it doesn't do anything.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
ff6db4b58c tools: ynl-gen: enable code gen for directional specs
I think that user space code gen for directional specs
works after recent changes. Let them through.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
6f115d4575 tools: ynl-gen: refactor strmap helper generation
Move generating strmap lookup function to a helper.
No functional changes.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
9858bfc271 tools: ynl-gen: use enum names in op strmap more carefully
In preparation for supporting families which use different msg
ids to and from the kernel - make sure the ids in op strmap
are correct. The map is expected to be used mostly for notifications,
don't generate a separate map for the "to kernel" direction.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-08 14:01:10 -07:00
Jakub Kicinski
7a11f70ce8 tools: ynl: generate code for the handshake family
Generate support for the handshake family.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
58da455b31 tools: ynl-gen: improve unwind on parsing errors
When parsing multi-attr we count the objects and then allocate
an array to hold the parsed objects. If an attr space has multiple
multi-attr objects, however, if parsing the first array fails
we'll leave the object count for the second even tho the second
array was never allocated.

This may cause crashes when freeing objects on error.

Count attributes to a variable on the stack and only set the count
in the object once the memory was allocated.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
2cc9671a82 tools: ynl-gen: fill in support for MultiAttr scalars
The handshake family needs support for MultiAttr scalars.
Right now we only support code gen for MultiAttr nested
types.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-07 12:53:10 -07:00
Jakub Kicinski
ee0202e2e7 tools: ynl: add sample for netdev
Add a sample application using the C library.
My main goal is to make writing selftests easier but until
I have some of those ready I think it's useful to show off
the functionality and let people poke and tinker.

Sample outputs - dump:

$ ./netdev
Select ifc ($ifindex; or 0 = dump; or -2 ntf check): 0
      lo[1]	0:
  enp1s0[2]	23: basic redirect rx-sg

Notifications (watching veth pair getting added and deleted):

$ ./netdev
Select ifc ($ifindex; or 0 = dump; or -2 ntf check): -2
[53]	0: (ntf: dev-add-ntf)
[54]	0: (ntf: dev-add-ntf)
[54]	23: basic redirect rx-sg (ntf: dev-change-ntf)
[53]	23: basic redirect rx-sg (ntf: dev-change-ntf)
[53]	23: basic redirect rx-sg (ntf: dev-del-ntf)
[54]	23: basic redirect rx-sg (ntf: dev-del-ntf)

Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:32 -07:00
Jakub Kicinski
d75fdfbc6f tools: ynl: support fou and netdev in C
Generate the code for netdev and fou families. They are simple
and already supported by the code gen.

Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:32 -07:00
Jakub Kicinski
86878f14d7 tools: ynl: user space helpers
Add "fixed" part of the user space Netlink Spec-based library.
This will get linked with the protocol implementations to form
a full API.

Acked-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:31 -07:00
Jakub Kicinski
a99bfdf647 tools: ynl-gen: clean up stray new lines at the end of reply-less requests
Do not print empty lines before closing brackets.

Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-06 12:31:31 -07:00
Jakub Kicinski
59d814f0f2 tools: ynl-gen: generate static descriptions of notifications
Notifications may come in at any time. The family must be always
ready to parse a random incoming notification. Generate notification
table for parsing and tell YNL which request we're processing
to distinguish responses from notifications.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
8cb6afb335 tools: ynl-gen: switch to family struct
We'll want to store static info about the family soon.
Generate a struct. This changes creation from, e.g.:

	 ys = ynl_sock_create("netdev", &yerr);
to:
	 ys = ynl_sock_create(&ynl_netdev_family, &yerr);

on user's side.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5d58f911c7 tools: ynl-gen: generate alloc and free helpers for req
We expect user to allocate requests with calloc(),
make things a bit more consistent and provide helpers.
Generate free calls, too.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
dc0956c98f tools: ynl-gen: move the response reading logic into YNL
We generate send() and recv() calls and all msg handling for
each operation. It's a lot of repeated code and will only grow
with notification handling. Call back to a helper YNL lib instead.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
21b6e30278 tools: ynl-gen: generate enum-to-string helpers
It's sometimes useful to print the name of an enum value,
flag or name of the op. Python can do it, add C helper
code gen for getting names of things.

Example:

  static const char * const netdev_xdp_act_strmap[] = {
	[0] = "basic",
	[1] = "redirect",
	[2] = "ndo-xmit",
	[3] = "xsk-zerocopy",
	[4] = "hw-offload",
	[5] = "rx-sg",
	[6] = "ndo-xmit-sg",
  };

  const char *netdev_xdp_act_str(enum netdev_xdp_act value)
  {
	value = ffs(value) - 1;
	if (value < 0 || value >= (int)MNL_ARRAY_SIZE(netdev_xdp_act_strmap))
		return NULL;
	return netdev_xdp_act_strmap[value];
  }

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
eef9b794ea tools: ynl-gen: add error checking for nested structs
Parsing nested types may return an error, propagate it.
Not marking as a fix, because nothing uses YNL upstream.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
5605f10237 tools: ynl-gen: loosen type consistency check for events
Both event and notify types are always consistent. Rewrite
the condition checking if we can reuse reply types to be
less picky and let notify thru.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:47 -07:00
Jakub Kicinski
67c65ce762 tools: ynl-gen: don't override pure nested struct
For pure structs (parsed nested attributes) we track what
forms of the struct exist in request and reply directions.
Make sure we don't overwrite the recorded struct each time,
otherwise the information is lost.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
6ad49839ba tools: ynl-gen: fix unused / pad attribute handling
Unused and Pad attributes don't carry information.
Unused should never exist, and be rejected.
Pad should be silently skipped.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
91dfaef243 tools: ynl-gen: add extra headers for user space
Make sure all relevant headers are included, we allocate memory,
use memcpy() and Linux types without including the headers.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-02 22:10:46 -07:00
Jakub Kicinski
a03a91bd68 Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Cross-merge networking fixes after downstream PR.

No conflicts.

Adjacent changes:

drivers/net/ethernet/sfc/tc.c
  622ab65634 ("sfc: fix error unwinds in TC offload")
  b6583d5e9e ("sfc: support TC decap rules matching on enc_src_port")

net/mptcp/protocol.c
  5b825727d0 ("mptcp: add annotations around msk->subflow accesses")
  e76c8ef5cc ("mptcp: refactor mptcp_stream_accept()")

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-06-01 15:38:26 -07:00
Donald Hunter
313a7a808c tools: ynl: Support enums in struct members in genetlink-legacy
Support decoding scalars as enums in struct members for genetlink-legacy
specs.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-05-29 22:05:38 -07:00
Donald Hunter
5ac18889bd tools: ynl: Initialise fixed headers to 0 in genetlink-legacy
This eliminates the need for e.g. --json '{"dp-ifindex":0}' which is not
too big a deal for ovs but will get tiresome for fixed header structs that
have many members.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-05-29 22:05:38 -07:00
Jakub Kicinski
9b66ee06e5 net: ynl: prefix uAPI header include with uapi/
To keep things simple we used to include the uAPI header
in the kernel in the #include <linux/$family.h> format.
This works well enough, most of the genl families should
have headers in include/net/ so linux/$family.h ends up
referring to the uAPI header, anyway. And if it doesn't
no big deal, we'll just include more info than we need.

Unless that is there is a naming conflict. Someone recently
created include/linux/psp.h which will be a problem when
supporting the PSP protocol. (I'm talking about
work-in-progress patches, but it's just a proof that assuming
lack of name conflicts was overly optimistic.)

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-05-26 10:30:14 +01:00
Jakub Kicinski
081e8df681 tools: ynl: avoid dict errors on older Python versions
Python 3.9.0 or newer supports combining dicts() with |,
but older versions of Python are still used in the wild
(e.g. on CentOS 8, which goes EoL May 31, 2024).
With Python 3.6.8 we get:

  TypeError: unsupported operand type(s) for |: 'dict' and 'dict'

Use older syntax. Tested with non-legacy families only.

Fixes: f036d936ca ("tools: ynl: Add fixed-header support to ynl")
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Tested-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20230524170712.2036128-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-05-25 21:02:43 -07:00
Donald Hunter
bddd2e561b tools: ynl: Handle byte-order in struct members
Add support for byte-order in struct members in the genetlink-legacy
spec.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-05-24 08:46:54 +01:00
Donald Hunter
7c2435ef76 tools: ynl: Use dict of predefined Structs to decode scalar types
Use a dict of predefined Struct() objects to decode scalar types in native,
big or little endian format. This removes the repetitive code for the
scalar variants and ensures all the signed variants are supported.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2023-05-24 08:46:54 +01:00
Rahul Rameshbabu
85a4abed15 tools: ynl: Rename ethtool to ethtool.py
Make it explicit that this tool is not a drop-in replacement for ethtool.
This tool is intended for testing ethtool functionality implemented in the
kernel and should use a name that differentiates it from the ethtool
utility.

Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Link: https://lore.kernel.org/r/20230413012252.184434-2-rrameshbabu@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-04-13 22:18:29 -07:00
Rahul Rameshbabu
3ea31e6664 tools: ynl: Remove absolute paths to yaml files from ethtool testing tool
Absolute paths for the spec and schema files make the ethtool testing tool
unusable with freshly checked-out source trees. Replace absolute paths with
relative paths for both files in the Documentation/ directory.

Issue seen before the change

  Traceback (most recent call last):
    File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/./ethtool", line 424, in <module>
      main()
    File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/./ethtool", line 158, in main
      ynl = YnlFamily(spec, schema)
    File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/lib/ynl.py", line 342, in __init__
      super().__init__(def_path, schema)
    File "/home/binary-eater/Documents/mlx/linux/tools/net/ynl/lib/nlspec.py", line 333, in __init__
      with open(spec_path, "r") as stream:
  FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/google/home/sdf/src/linux/Documentation/netlink/specs/ethtool.yaml'

Fixes: f3d07b02b2 ("tools: ynl: ethtool testing tool")
Signed-off-by: Rahul Rameshbabu <rrameshbabu@nvidia.com>
Link: https://lore.kernel.org/r/20230413012252.184434-1-rrameshbabu@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2023-04-13 22:18:29 -07:00