pidl s4::NDR::Parser: read hex numbers as numbers for ranges
Hex numbers in IDL are not parsed as numbers, resulting in warnings
like
Argument 0x2000 isn't numeric in numeric lt (<) at /home/douglas/src/samba/pidl/lib/Parse/Pidl/Samba4/NDR/Parser.pm line 981
not to mention problematic code. We add a utility function to convert
these numbers to numbers.
A typical difference this makes is:
--- old/default/librpc/gen_ndr/ndr_dcerpc.c 2019-11-30 23:40:32.
915816967 +1300
+++ new/default/librpc/gen_ndr/ndr_dcerpc.c 2019-11-30 17:00:09.
055733660 +1300
@@ -1893,7 +1893,7 @@
if (ndr_flags & NDR_SCALARS) {
NDR_CHECK(ndr_pull_align(ndr, 4));
NDR_CHECK(ndr_pull_uint32(ndr, NDR_SCALARS, &r->ReceiveWindowSize));
- if (r->ReceiveWindowSize > 0x40000) {
+ if (r->ReceiveWindowSize < 8192 || r->ReceiveWindowSize > 262144) {
return ndr_pull_error(ndr, NDR_ERR_RANGE, "value out of range");
}
NDR_CHECK(ndr_pull_trailer_align(ndr, 4));
Where the minimum ("0x2000" == 8192) was read as a string, thus
treated as zero.
The treatment as zero was introduced in
142b2a61f8a77b3065ce4c78b459ab714d6d190a
accidentially, which shows why warnings are important.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>