Update for the KDL 1.0 spec (changes ported across from kdl4j with enhancements)

* Ported KDL1.0 changes across but unit tests are broken

* Port a minor syntax fix from kdl4j

* Fix BOM on unit tests/KDL output, and import a whole ton of example testcases from kdl4j

* import all testcases from main KDL repo

* Fix parsing booleans with type annotations, and some compiler warnings

* Split out KDL types into their own files

* Implement BigInteger support for KDL with radix preservation

* More test compliance around string escaping

* Radix stuff for conformance testing

* more numeric compliance

* Final compliance tests. Extremely large/small scientific notation values are out of scope for KDL.NET at this time

* Update README.md

* Update KDL config for KdlDotNet1.1.0

* Create .gitattributes

Author: borland

.gitattributes

@@ -0,0 +1 @@
+* -text

KdlDotNet.Extensions.Configuration/src/KdlDotNet.Extensions.Configuration.csproj

@@ -6,20 +6,20 @@
     <LangVersion>8.0</LangVersion>
     <AssemblyName>KdlDotNet.Extensions.Configuration</AssemblyName>
 
-    <PackageVersion>5.0.1</PackageVersion>
-    <AssemblyVersion>5.0.1</AssemblyVersion>
-    <FileVersion>5.0.1</FileVersion>
-    <Version>5.0.1</Version>
+    <PackageVersion>5.1.0</PackageVersion>
+    <AssemblyVersion>5.1.0</AssemblyVersion>
+    <FileVersion>5.1.0</FileVersion>
+    <Version>5.1.0</Version>
     <Authors>Orion Edwards</Authors>
     <Company />
     <PackageLicenseExpression>CC-BY-4.0</PackageLicenseExpression>
     <PackageProjectUrl>https://github.com/borland/kdl-net</PackageProjectUrl>
     <Description>ASP.NET Core Configuration provider for KDL files; .NET 5 distribution</Description>
-    <Copyright>Orion Edwards 2020</Copyright>
+    <Copyright>Orion Edwards 2021</Copyright>
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="KdlDotNet" Version="1.0.1" />
+    <PackageReference Include="KdlDotNet" Version="1.1.0" />
     <PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="5.0.0" />
     <PackageReference Include="Microsoft.Extensions.FileProviders.Abstractions" Version="5.0.0" />
     <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="5.0.0" />

README.md

@@ -5,6 +5,15 @@ This is a C# implementation of a parser for the [KDL Document Language](https://
 It is semi-literally ported from the Java KDL parser implementation [kdl4j](https://github.com/hkolbeck/kdl4j) by [Hannah Kolbeck](https://github.com/hkolbeck).
 Many thanks for the original implementation.
 
+## Status
+
+**The library is up to date as of the KDL 1.0.0. spec.** All testcases pass, except for scientific-notation numeric values outside the range representable by a `System.Double`, which KdlDotNet does not support at this time.
+
+The scope of the library is small and it has what I would consider excellent unit test coverage.
+
+I would be happy to use it in production in a constrained environment, such as loading configuration files, etc. It hasn't had proper performance benchmarking
+or profiling, so it is probably not suitable for use in a performance-critical codepath such as a tight loop.
+
 ## Platform
 
 The library is built against .NET Standard 2.0, which means it should work on the .NET Desktop framework 4.7.2 or later, and .NET Core 2.0 and later. The unit tests run against both .NET 4.7.2 and .NET Core 3.1 and all passs.
@@ -17,15 +26,6 @@ Install the [KdlDotNet Nuget package](https://www.nuget.org/packages/KdlDotNet),
 
 KDL-net is licensed under the Creative Commons Attribution 4.0 License.
 
-## Status
-
-The scope of the library is small and it has what I would consider excellent unit test coverage.
-
-I would be happy to use it in production in a constrained environment, such as loading configuration files, etc. It hasn't had any proper performance benchmarking
-or profiling, so it is probably not suitable for use in a performance-critical codepath such as a tight loop.
-
-One Caveat: The KDL language spec itself is not final and may still yet change so I can't in all honesty say this is a production library. In my personal view, the spec looks pretty solid, and if it does change, I wouldn't expect it to do so in a major breaking way; rather I'd expect simple clarification of edge cases and bug-fixes. Hopefully nothing dramatic.
-
 ## Why KDL?
 
 I have successfully used this library to configure and boot up an ASP.NET Core application, using KDL in place of what would usually be a JSON file. KDL fits very nicely in this kind of environment. I've created an extension library Nuget Package called [KdlDotNet.Extensions.Configuration](https://www.nuget.org/packages/KdlDotNet.Extensions.Configuration/) to help with this.
@@ -89,7 +89,7 @@ If you want to do something other than configure asp.net core applications, you'
 Add a nuget package reference to the [KdlDotNet](https://www.nuget.org/packages/KdlDotNet/) package
 
 ```xml
-<PackageReference Include="KdlDotNet" Version="1.0.0" />
+<PackageReference Include="KdlDotNet" Version="1.1.0" />
 ```
 
 Then create a new instance of `KdlDotNet.KDLParser` and call the `Parse` method, either on a `Stream` if you have one from a file/network, or from a `string`.  
@@ -113,8 +113,10 @@ kdl4j uses the java BigDecimal type as the internal representation for all numbe
 Presumably this was done for ease of implementation as it lets kdl4j have a simpler code path for handling numbers, 
 however BigDecimal is a large and complex thing, weighing in at a minimum of 32 bytes per instance (over and above the surrounding KDLNumber object).
 
-kdl-net in contrast uses an 8 byte storage structure, which is a union containing either an int32, int64, or double (8 byte floating point) value.
-This means kdl-net can't handle numbers that exceed the size of an int64 or double, however it should require a lot less memory and be faster.
-Given that the primary use-case for KDL at this stage seems to be human-readable configuration, this seems like a better tradeoff.
+KdlDotNet in contrast uses a hierarchy of classes derived from an abstract `KDLNumber` class, each of which uses different backing storage.
+When parsing KDL documents, KdlDotNet will attempt to use the smallest integer type in the series of `Int32`, `Int64`, `BigInteger`.
+All floating point or scientific-notation values are stored using `Double`.
+
+This means KdlDotNet should require a lot less memory and be faster when dealing with numeric values.
 
 **PULL REQUESTS APPRECIATED**

src/CharClasses.cs

@@ -46,15 +46,18 @@ public static bool IsValidNumericStart(int c)
          */
         public static bool IsValidBareIdChar(int c)
         {
+            if (c <= 0x20 || c > 0x10FFFF)
+                return false;
+
             switch (c)
             {
-                case '\n':
-                case '\u000C':
-                case '\r':
                 case '\u0085':
                 case '\u2028':
                 case '\u2029':
                 case '\\':
+                case '/':
+                case '(':
+                case ')':
                 case '{':
                 case '}':
                 case '<':
@@ -65,8 +68,6 @@ public static bool IsValidBareIdChar(int c)
                 case '=':
                 case ',':
                 case '"':
-                case '\u0009':
-                case '\u0020':
                 case '\u00A0':
                 case '\u1680':
                 case '\u2000':
@@ -95,15 +96,41 @@ public static bool IsValidBareIdChar(int c)
          * @param c the character to check
          * @return true if the character is valid, false otherwise
          */
-        public static bool IsValidBareIdStart(int c) => !IsValidDecimalChar(c) && IsValidBareIdChar(c);
+        public static bool IsValidBareIdStart(int c)
+             => !IsValidDecimalChar(c) && IsValidBareIdChar(c);
+
+        /**
+         * Check if a string is a valid bare identifier
+         *
+         * @param string the string to check
+         * @return true if the string is a valid bare id, false otherwise
+         */
+        public static bool IsValidBareId(string str)
+        {
+            if (str == "")
+                return false;
+
+            var validBareIdStart = IsValidBareIdStart(str[0]);
+            if (str.Length == 1 || !validBareIdStart)
+                return validBareIdStart;
+
+            for (int i = 0; i < str.Length; i++)
+            {
+                if (!IsValidBareIdChar(str[i]))
+                    return false;
+            }
+
+            return true;
+        }
 
         /**
          * Check if the character is a valid decimal digit
          *
          * @param c the character to check
          * @return true if the character is valid, false otherwise
          */
-        public static bool IsValidDecimalChar(int c) => ('0' <= c && c <= '9');
+        public static bool IsValidDecimalChar(int c)
+            => ('0' <= c && c <= '9');
 
         /**
          * Check if the character is a valid hexadecimal digit
@@ -149,15 +176,17 @@ public static bool IsValidHexChar(int c)
          * @param c the character to check
          * @return true if the character is valid, false otherwise
          */
-        public static bool IsValidOctalChar(int c) => ('0' <= c && c <= '7');
+        public static bool IsValidOctalChar(int c)
+            => ('0' <= c && c <= '7');
 
         /**
          * Check if the character is a valid binary digit
          *
          * @param c the character to check
          * @return true if the character is valid, false otherwise
          */
-        public static bool IsValidBinaryChar(int c) => (c == '0' || c == '1');
+        public static bool IsValidBinaryChar(int c)
+            => (c == '0' || c == '1');
 
         /**
          * Check if the character is contained in one of the three literal values: true, false, and null
@@ -247,9 +276,13 @@ public static bool IsUnicodeWhitespace(int c)
          * @return true if the character is printable unescaped, false otherwise
          */
         public static bool IsPrintableAscii(int c)
-        {
-            return ' ' <= c && c <= '~' && c != '/' && c != '"';
-        }
+            => (' ' <= c && c <= '~');
+
+        public static bool IsNonAscii(int c)
+            => c > 127;
+
+        public static bool MustEscape(int c)
+            => c == '\\' || c == '"';
 
         private static readonly string ESC_BACKSLASH = "\\\\";
         private static readonly string ESC_BACKSPACE = "\\b";
@@ -266,7 +299,8 @@ public static bool IsPrintableAscii(int c)
          * @param c the character to check
          * @return An Optional wrapping the escape sequence string if the character needs to be escaped, or false otherwise
          */
-        public static string? GetCommonEscape(int c) =>  c switch {
+        public static string? GetCommonEscape(int c) => c switch
+        {
             '\\' => ESC_BACKSLASH,
             '\b' => ESC_BACKSPACE,
             '\n' => ESC_NEWLINE,
@@ -278,12 +312,30 @@ public static bool IsPrintableAscii(int c)
             _ => null,
         };
 
+        public static bool IsCommonEscape(int c)
+        {
+            switch (c)
+            {
+                case '\\':
+                case '\b':
+                case '\n':
+                case '\f':
+                case '\t':
+                case '\r':
+                case '"':
+                    return true;
+                default:
+                    return false;
+            }
+        }
+
         /**
          * Get the escape sequence for any character
          *
          * @param c the character to check
          * @return The escape sequence string
          */
-        public static string GetEscapeIncludingUnicode(int c) => GetCommonEscape(c) ?? string.Format("\\u{0:x}", c);
+        public static string GetEscapeIncludingUnicode(int c)
+            => GetCommonEscape(c) ?? $"\\u{c:x}";
     }
 }

src/KDLBoolean.cs

@@ -0,0 +1,53 @@
+using System.IO;
+
+#nullable enable
+
+namespace KdlDotNet
+{
+    /**
+     * A KDL object holding a boolean value. New instances should not be created, instead use the TRUE or FALSE constants
+     */
+    public class KDLBoolean : KDLValue<bool>
+    {
+        public static KDLBoolean? FromString(string str, string? type) => str switch
+        {
+            "true" => type == null ? True : new KDLBoolean(true, type),
+            "false" => type == null ? False : new KDLBoolean(false, type),
+            _ => null,
+        };
+
+        public static KDLBoolean From(bool b, string? type = null)
+            => type == null ?
+                b ? True : False :
+                new KDLBoolean(b, type);
+
+        public static KDLBoolean True { get; } = new KDLBoolean(true);
+        public static KDLBoolean False { get; } = new KDLBoolean(false);
+
+        private static readonly KDLString trueStr = KDLString.From("true");
+        private static readonly KDLString falseStr = KDLString.From("false");
+
+        public KDLBoolean(bool value) : this(value, null) { }
+
+        public KDLBoolean(bool value, string? type) : base(type)
+            => Value = value;
+
+        public override bool Value { get; }
+
+        public override KDLString AsString() => Value ? trueStr : falseStr;
+        public override KDLNumber? AsNumber() => null;
+        public override KDLBoolean? AsBoolean() => this;
+
+        protected override void WriteKDLValue(StreamWriter writer, PrintConfig printConfig)
+            => writer.Write(Value ? "true" : "false");
+
+        public override bool IsBoolean => true;
+
+        public override string ToString() => $"KDLBoolean{{value='{Value}', type={Type}}}";
+
+        public override bool Equals(object? obj)
+            => obj is KDLBoolean other && other.Value == Value && Type == other.Type;
+
+        public override int GetHashCode() => Value.GetHashCode();
+    }
+}

src/KDLDocument.cs

@@ -62,7 +62,7 @@ public void WriteKDLPretty(StreamWriter writer) // throws IOException
         public string ToKDLPretty(PrintConfig printConfig)
         {
             var writer = new MemoryStream();
-            var bufferedWriter = new StreamWriter(writer, Encoding.UTF8);
+            var bufferedWriter = new StreamWriter(writer, KDLObjectExtensions.UTF8NoByteOrderMarkerEncoding); // no BOM
 
             WriteKDLPretty(bufferedWriter, printConfig);
             bufferedWriter.Flush();