• Avalanche
  • Avalanche.Core
  • Lexical

    Show / Hide Table of Contents
    • Avalanche.Localization
      • Introduction
      • Localization
      • LocalizationFile
      • LocalizationFiles
      • LocalizationFileSystem
      • LocalizationFileFormat
      • LocalizationLine
      • LocalizationLines
      • TemplateFormat
      • CultureProvider
      • FallbackCultureProvider
      • ResourceManager
      • LocalizationError
      • Microsoft.Extensions
        • Introduction
        • DependencyInjection
        • FileProvider
        • Logging
        • ITextLocalizer
        • IFileLocalizer
        • Localization
        • AspNetCore
      • Pluralization
        • Introduction
        • Multiple plural parameters
        • Custom PluralRules
        • Invariant Culture
        • Unit Prefix
        • IPluralRule
        • IPluralNumber
        • IPluralRules
        • CLDRs
        • Unicode.CLDR41
        • Unicode.CLDR40
      • Articles
        • Alphabet localization
        • Benchmarks
        • Caching
        • Class Library
        • Demo
        • Diagnostics
        • Embedded resources
        • Emplacement
        • File localization
        • Text localization
    • Avalanche.Message
      • Introduction
      • IMessage
      • IMessageProvider
      • IMessageDescription
      • IMessageDescriptions
      • MessageLevel
      • Message printing
      • Messages and Exceptions
      • Logging
      • Validation
      • Localization
    • Avalanche.StatusCode
      • Introduction
      • HResult
        • Introduction
        • HResult.Facilities
        • BasicMessages
        • RpcMessages
        • DispatchMessages
        • ItfMessages
        • Win32Messages
        • ClrMessages
      • System
        • Introduction
        • AccessControlMessages
        • AggregateMessages
        • AppDomainMessages
        • ArgumentMessages
        • ArgumentNullMessages
        • ArgumentOutOfRangeMessages
        • ArithmeticMessages
        • ArrayMessages
        • AssemblyMessages
        • BadImageFormatMessages
        • CodeContractMessages
        • CodePageMessages
        • CollectionsMessages
        • CompilerServiceMessages
        • CryptographyMessages
        • CultureMessages
        • DiagnosticsMessages
        • EventSourceMessages
        • ExecutionEngineMessages
        • FormatMessages
        • HostProtectionMessages
        • IOMessages
        • IndexOutOfRangeMessages
        • InteropServiceMessages
        • InvalidCastMessages
        • InvalidOperationMessages
        • IsolatedStorageMessages
        • LazyMessages
        • MarshalerMessages
        • MemoryMessages
        • MiscellaneousMessages
        • NotImplementedMessages
        • NotSupportedMessages
        • ObjectDisposedMessages
        • OperationCanceledMessages
        • OverflowMessages
        • PlatformMessages
        • PolicyMessages
        • PrincipalMessages
        • ProgramMessages
        • ReferenceMessages
        • ReflectionMessages
        • RegionMessages
        • RemotingMessages
        • ResourcesMessages
        • SecurityMessages
        • SerializationMessages
        • StackMessages
        • TaskMessages
        • TextMessages
        • ThreadingMessages
        • TimeZoneMessages
        • TypeMessages
        • XmlMessages
    • Avalanche.Template
      • Introduction
      • TemplateFormats
      • ITemplatePrintable
      • ITemplateFormatPrintable
      • ITemplateText
      • ITemplateBreakdown
      • ITemplateFormat
      • ITemplateFormats
      • Extract Arguments
      • Emplacement
    • Avalanche.Tokenizer
      • Introduction
      • IToken
      • ITokenizer
      • Tokenizers
    • Avalanche.Service
      • Introduction
      • Service
        • IService
        • Construction
        • Query
        • CancellationToken
        • CachePolicy
        • Cast
        • Scope
        • Dispose
      • Handler
        • Introduction
        • IHandler
        • Cancel
        • Recursion
        • Cyclicity
        • Delegates
        • Casting
        • Invokable
        • Handlers
      • Request
        • Introduction
        • IRequest
        • [Request]
        • [ContextParameter]
        • Print Tree
      • Dependency Injection
        • Introduction
        • ServiceRequest<T>
        • Handler
        • CachePolicy
        • CancellationToken
        • QueryLogger
      • Examples
        • Expression Example
        • Mapper Example
      • ServiceMessages
    • Avalanche.Writer
      • Introduction
      • Writer
      • Referer
      • TypeCast
      • DefaultConstructor
      • DelegateWriter
      • WriterPipe
      • ConstantWriter
      • PassthroughWriter
      • Context
      • WriterMessages
    • Avalanche.Accessor
      • Introduction
      • AccessorServices
      • List
        • ListAccessor
        • ListConstructor
        • ListDeconstructor
        • ListDefaultConstructor
        • ListGet
        • ListSet
        • ListAdd
        • ListInsert
        • ListClear
        • ListCount
        • ListIndexOf
        • ListRemoveAt
        • ListReferer
      • Map
        • MapAccessor
        • MapConstructor
        • MapDeconstructor
        • MapDefaultConstructor
        • MapSet
        • MapGet
        • MapRemove
        • MapClear
        • MapCount
        • MapReferer
      • Record
        • RecordAccessor
        • RecordConstructor
        • RecordDeconstructor
        • RecordDefaultConstructor
      • Field
        • FieldAccessor
        • FieldReader
        • FieldWriter
        • FieldReferer
      • Dependency Injection
        • Introduction
      • AccessorMessages
    • Avalanche.Emit
      • Introduction
      • TypeBuilder
      • ConstructorBuilder
      • MethodBuilder
      • PropertyBuilder
      • FieldBuilder
      • Emit
      • Utilities
    • Avalanche.DataType
      • Introduction
      • IDataTypeBase
      • IDataType
      • IRecordType
      • IFieldType
      • IUnionType
      • IListType
      • IMapType
      • IValueType
      • IEnumerationType
      • IIntegerType
      • IRealType
      • IStringType
      • IAnyType
      • DataTypeRequest
      • PrintTree
      • DataTypeMessages
    • Avalanche.FileSystem
      • Introduction
      • Abstractions
        • IFileSystem
          • IFileSystemBrowse
          • IFileSystemCreateDirectory
          • IFileSystemDelete
          • IFileSystemFileAttribute
          • IFileSystemMount
          • IFileSystemMove
          • IFileSystemObserve
          • IFileSystemOpen
        • IEvent
        • IEntry
        • IOption
        • IToken
      • FileSystem
      • VirtualFileSystem
      • MemoryFileSystem
      • EmbeddedFileSystem
      • HttpFileSystem
      • Decoration
      • IFileProvider
      • Events
      • Utilities
        • Dispose
        • File Scanner
        • Visit Tree
        • File Operation
    • Avalanche.Identity
      • Introduction
      • Identity
      • IdentityParts
      • IdentityInterner
      • IdentityComparer
      • Print Tree
      • IdentityAccessors
        • Introduction
        • TypeName
    • Avalanche.Utilities
      • Introduction
      • Collections
        • Tuples
        • StructList
        • ArrayList
        • BijectionMap
        • LocakableDictionary
        • LockableList
        • MapList
        • Pipe
        • RingQueue
        • EnumerableExtensions
        • TupleUtilities
        • ArrayUtilities
      • Comparers
        • IGraphComparer
        • IGraphComparable
        • AlphaNumericComparer
        • EnumerableComparer
        • EnumerableGraphComparer
        • ReferenceComparer
        • KeyValuePairComparer
        • DefaultComparerProvider
        • RecordComparer
      • Cloners
        • ICloner
        • IGraphCloner
        • IGraphCloneable
        • EnumerableCloner
        • FieldCloner
        • PassthroughCloner
        • RecordCloner
        • ClonerProvider
      • Dispose
        • IDisposeAttachable
        • IDisposeBelatable
      • Provider
        • Introduction
        • ProviderBase
        • Delegate
        • Concatenation
        • Cache
        • ResultCapture
        • AsReadOnly
        • AsService
      • Record
        • IRecordDescription
        • IFieldDescription
        • IConstructorDescription
        • IConstructionDescription
        • IParameterDescription
        • IRecordProviders
        • RecordDelegates
          • RecordCreate
          • RecordClone
          • RecordCopy
          • IRecordDelegates
        • FieldDelegates
          • FieldRead
          • FieldWrite
          • RecreateWith
          • IFieldDelegates
      • String
        • IEscaper
        • UnicodeString
      • Miscellaneous
        • IIdGenerator
        • Permutation
        • IReadOnly
        • IUserDataContainer
        • Void
    • Avalanche.Core
      • License
      • Order

    UnicodeString

    UnicodeString addresses issues that come when handing strings with different encodings.

    Construct

    UnicodeString can be constructed from

    • IEnumerable<byte>, byte[]
    • IEnumerable<char>, char[], String
    • IEnumerable<int>., int[]

    UnicodeString is a wrapper that keeps reference to the source object that it was constructed from.

    // Construct from String, IEnumerable<byte>, IEnumerable<char>, IEnumerable<int>
    UnicodeString str = new UnicodeString("European castle \uD83C\uDFF0");
    UnicodeString str8 = new UnicodeString(new byte[] { 69, 117, 114, 111, 112, 101, 97, 110, 32, 99, 97, 115, 116, 108, 101, 32, 240, 159, 143, 176 });
    UnicodeString str16 = new UnicodeString(new char[] { 'E', 'u', 'r', 'o', 'p', 'e', 'a', 'n', ' ', 'c', 'a', 's', 't', 'l', 'e', ' ', '\uD83C', '\uDFF0' });
    UnicodeString str32 = new UnicodeString(new int[] { 69, 117, 114, 111, 112, 101, 97, 110, 32, 99, 97, 115, 116, 108, 101, 32, 127984 });
    

    Uniform use

    Regardless of what source object is, as long as the encoding describes same unicode codepoints, UnicodeString describes the same uniform content.

    Console.WriteLine($"Src={str8.Source.GetType().Name},  Str=\"{str8}\", Length={str8.Length}, Hashcode={str8.GetHashCode()}");
    Console.WriteLine($"Src={str16.Source.GetType().Name},  Str=\"{str16}\", Length={str16.Length}, Hashcode={str16.GetHashCode()}");
    Console.WriteLine($"Src={str32.Source.GetType().Name}, Str=\"{str32}\", Length={str32.Length}, Hashcode={str32.GetHashCode()}");
    

    Encoding

    Stack allocated IEnumerables and IEnumerators to UTF8, UTF16 and UTF32 encodings can be used.

    // Encode to utf-8, utf-16, utf-32 codepoints
    foreach (byte codepoint in str) { }
    foreach (char codepoint in str) { }
    foreach (int codepoint in str) { }
    
    // Get stack allocated enumerator encoders
    for (UTF8Enumerator enumr = str8.GetEnumeratorUTF8(); enumr.MoveNext();) { }
    for (UTF16Enumerator enumr = str8.GetEnumeratorUTF16(); enumr.MoveNext();) { }
    for (UTF32Enumerator enumr = str8.GetEnumerator(); enumr.MoveNext();) { }
    
    // Length to different encodings can be calculated
    int utf8length = str8.UTF8Length;
    int utf16length = str8.UTF16Length;
    int utf32length = str8.Length;
    

    And converted to arrays of different encodings.

    // Convert to utf 8/16/32 codepoints
    Console.WriteLine($"Utf8 [ {String.Join(", ", str.ToUtf8Array())} ]");
    Console.WriteLine($"Utf16 [ {String.Join(", ", str.ToUtf16Array())} ]");
    Console.WriteLine($"Utf32 [ {String.Join(", ", str.ToArray())} ]");
    

    Hash-Equals

    UnicodeStrings can be compared for unicode equality.

    // Compare for unicode equality and hashcode
    var equals1 = str8.Equals(str32);
    var equals2 = str16.Equals(str8);
    
    var hashcode_equal1 = str8.GetHashCode() == str16.GetHashCode();
    var hashcode_equal2 = str16.GetHashCode() == str8.GetHashCode();
    

    Assign to IList

    UnicodeString can be assigned to

    • IEnumerable<byte>, IList<byte>
    • IEnumerable<char>, IList<char>
    • IEnumerable<int>, IList<int>
    // Assign UTF8 to ILists of various encodings
    IList<byte> utf8_to_utf8 = str8;
    IList<char> utf8_to_utf16 = str8;
    IList<int> utf8_to_utf32 = str8;
    
    // Assign UTF16 to ILists of various encodings
    IList<byte> utf16_to_utf8 = str16;
    IList<char> utf16_to_utf16 = str16;
    IList<int> utf16_to_utf32 = str16;
    
    // Assign UTF32 to ILists of various encodings
    IList<byte> utf32_to_utf8 = str32;
    IList<char> utf32_to_utf16 = str32;
    IList<int> utf32_to_utf32 = str32;
    
    Back to top Copyright © 2022 Toni Kalajainen, contact@avalanche.fi