GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - marpiec/mpjsons: Marcin Pieciukiewicz's JSON library for Scala.

BranchesTags
Open more actions menu

Latest commit

 

History

283 Commits
283 Commits

Folders and files

NameName
Last commit message
Last commit date
bench-results
bench-results
 
 
docs
docs
 
 
project
project
 
 
src
src
 
 
.gitattributes
.gitattributes
 
 
.gitignore
.gitignore
 
 
CLAUDE.md
CLAUDE.md
 
 
LICENSE.txt
LICENSE.txt
 
 
README.md
README.md
 
 
build.sbt
build.sbt
 
 

Repository files navigation

mpjsons

mpjsons is a fast, minimal-API object ⇄ JSON conversion library for Scala.

Its whole design goal is the simplest possible API: serialize any object to JSON and deserialize JSON back into a typed object with no hand-written serializers. Type information is discovered through Scala 2.13 runtime reflection, and the generated per-type (de)serializers are cached, so after warm-up conversion runs through tight, allocation-conscious loops.

import io.mpjsons.MPJsons

val mpjsons = new MPJsons

case class User(firstName: String, lastName: String)

val json = mpjsons.serialize(User("John", "Doe"))
// {"firstName":"John","lastName":"Doe"}

val user = mpjsons.deserialize[User](json)
// User(John,Doe)

For most use cases, serialize and deserialize are all you will ever need.


Features

  • Zero boilerplate — serialize/deserialize case classes and mutable beans directly from their fields.
  • Generics — List[User], Map[String, List[User]], and other nested type parameters are resolved automatically.
  • Collections — List, Vector, Seq, Set, Queue, Stream, Array, and Map, both mutable and immutable, including Hash*/List* variants.
  • Scala types — Option, Either, and Tuple1–Tuple6.
  • java.time and friends — LocalDate, LocalTime, LocalDateTime, Duration (objects, or ISO-8601 strings with isoTime), Instant, OffsetDateTime, ZonedDateTime, java.util.UUID (strings), BigDecimal and BigInt (exact numbers).
  • Custom converters — plug in your own (de)serializers for any type.
  • Polymorphic JSON — embed type tags so collections of subtypes round-trip (markTypedClass).
  • Helpful errors — deserialization failures are wrapped with a pointer to the exact spot in the JSON.
  • Hot-path helpers — reusable, single-type StaticSerializer/StaticDeserializer.

Requirements

  • Scala 2.13.16
  • Java 25 or newer (deserialization can fall back to sun.misc.Unsafe for constructor-less classes — see Limitations)

Installation

mpjsons is not published to Maven Central or any public repository. Build it from source and consume it locally.

Clone and compile:

git clone <repo-url>
cd mpjsons
sbt compile
sbt test        # optional: run the full spec suite

To use it from another project, enable local publishing. The publishLocal task is disabled in build.sbt (publishLocal := {}); remove that line, then:

sbt publishLocal

and add the dependency to your project:

libraryDependencies += "io.mpjsons" %% "mpjsons" % "0.7.0"

(Current version: 0.7.0, Scala 2.13.16. Runtime dependency: scala-reflect.)


Usage manual

Create one MPJsons instance and reuse it:

import io.mpjsons.MPJsons

val mpjsons = new MPJsons

The constructor takes two optional flags (both default to false):

new MPJsons(ignoreNonExistingFields = false, ignoreNullFields = false)
  • ignoreNonExistingFields — on deserialize, silently skip JSON fields that don't exist on the target type instead of throwing.
  • ignoreNullFields — on serialize, omit null fields instead of throwing.

Both are covered in detail below.

Case classes and beans

Case classes and mutable (var-based) beans both work. Deserialization mutates fields in place, so target types should be case classes or beans with var fields.

case class User(a: String)
mpjsons.serialize(User("Hello"))               // {"a":"Hello"}
mpjsons.deserialize[User]("""{"a":"Hello"}""") // User(Hello)

class InnerObject {
  var intValue: Int = _
  var stringValue: String = _
}
mpjsons.deserialize[InnerObject]("""{"intValue":10,"stringValue":"Hello"}""")

The parser is lenient about input: it tolerates whitespace and accepts field names with or without quotes.

mpjsons.deserialize[InnerObject](""" {  intValue : 10 , stringValue : "Hello" } """)

Note: JSON keys follow declaration order; fields inherited from a superclass come first.

Collections

Lists, vectors, sets, queues, streams, and arrays serialize as JSON arrays:

mpjsons.serialize(List("a", "b", "c"))   // ["a","b","c"]
mpjsons.serialize(Vector(1, 2, 3))       // [1,2,3]
mpjsons.serialize(Set[Long](1, 11, 111)) // [1,11,111]

You can serialize a top-level collection directly (no wrapper object needed):

val json = mpjsons.serialize(List("Zxy", "Xyz", "ZyX"))
val back = mpjsons.deserialize[List[String]](json)

Generics compose freely:

mpjsons.serialize(Map("group" -> List(User("Ann"), User("Bob"))))
mpjsons.deserialize[Map[String, List[User]]](json)

Maps

By design, a regular Map serializes as a JSON array of [key, value] pairs, not as a JSON object. This lets arbitrary (non-string) key types round-trip:

mpjsons.serialize(Map("a" -> "Ala", "k" -> "Kot"))
// [["a","Ala"],["k","Kot"]]

mpjsons.serialize(Map(1 -> 1224, 5 -> 5324))
// [[1,1224],[5,5324]]

If you need the standard string-keyed object form ({"a":"Ala","k":"Kot"}), it is not selected automatically. Wire it up with SimpleMapSerializer / SimpleMapDeserializer via a custom converter for your specific Map[String, T] type.

Option, Either, Tuples

// Option — present values are inlined, None is absent
Some("test")   // serialized inline
None

// Either
Left(false)    // round-trips as Left(false)
Right(2)       // round-trips as Right(2)

// Tuples serialize as JSON arrays
mpjsons.serialize(Tuple1("Hello"))                 // ["Hello"]
mpjsons.serialize(("Hello", 2))                    // ["Hello",2]
mpjsons.serialize(("Hello", 2, "World", 5, "Hey")) // ["Hello",2,"World",5,"Hey"]

java.time

Time types serialize as structured objects, not ISO-8601 strings:

mpjsons.serialize(LocalTime.of(12, 30, 17, 125))
// {"hour":12,"minute":30,"second":17,"nano":125}

mpjsons.serialize(LocalDate.of(2023, 7, 23))
// {"year":2023,"month":7,"day":23}

mpjsons.serialize(LocalDateTime.of(2023, 8, 23, 14, 30, 17, 125))
// {"date":{"year":2023,"month":8,"day":23},"time":{"hour":14,"minute":30,"second":17,"nano":125}}

mpjsons.serialize(java.time.Duration.ofSeconds(123, 456))
// {"seconds":123,"nanos":456}

mpjsons.deserialize[LocalDate]("""{"year":2023,"month":7,"day":23}""")

Recursive and deeply nested types

Self-referential types are supported (element serializers for recursive types are resolved lazily):

case class RecursiveType(subtypes: List[RecursiveType])

val obj = RecursiveType(List(RecursiveType(List(RecursiveType(Nil))), RecursiveType(Nil)))
mpjsons.deserialize[RecursiveType](mpjsons.serialize(obj)) == obj  // true

A cyclic object graph, where an object refers back to itself through its fields, cannot be written as JSON: serializing one fails with an IllegalArgumentException naming the type. For untrusted input, set MPJsonsConfig(maxDepth = ...) to bound how deeply nested a document may be.

Numbers

Standard integer and floating-point formats are parsed, including scientific notation:

case class WithNumber(a: Int, b: Double, c: Double)
mpjsons.deserialize[WithNumber]("""{"a":30,"b":1.2e4,"c":2.1e2}""")
// WithNumber(30, 12000.0, 210.0)

Ignoring unknown fields

By default, unknown JSON fields cause a JsonInnerException. Pass ignoreNonExistingFields = true to skip them:

case class TestObject(a: Boolean, b: Int)
val json = """{"a": true, "c": "Hello", "d": [1,2,3,4], "b": 3}"""

new MPJsons(ignoreNonExistingFields = true).deserialize[TestObject](json)
// TestObject(true, 3)  — c and d are ignored

new MPJsons().deserialize[TestObject](json)
// throws JsonInnerException

Field names in JSON

By default the JSON key is the Scala field name. @jsonName overrides it, which is the simplest way to make the output smaller or to match an external schema. It works on case class parameters, vals and vars directly, without a meta-annotation, on inherited fields, and on members of traits and superclasses (see "Annotations on traits and superclasses").

import io.mpjsons.jsonName

case class User(@jsonName("n") firstName: String, @jsonName("l") lastName: String, age: Int)

mpjsons.serialize(User("John", "Doe", 42))        // {"n":"John","l":"Doe","age":42}
mpjsons.deserialize[User]("""{"n":"John","l":"Doe","age":42}""")

On read, the Scala field name is still accepted as a fallback, so JSON written before a field was renamed keeps loading; if another field owns that name as its JSON name, the JSON name wins. Two fields of one type must not share a JSON name: that is reported when the (de)serializer is built.

Leaving fields out of the output

@skipSerialization leaves a field out of the JSON entirely: serialize does not write it and deserialize ignores its key.

import io.mpjsons.skipSerialization

case class Session(user: String, @skipSerialization token: String)

mpjsons.serialize(Session("ann", "secret"))                        // {"user":"ann"}
mpjsons.deserialize[Session]("""{"user":"ann","token":"secret"}""") // Session("ann", null)
mpjsons.deserialize[Session]("""{"user":"ann"}""")                  // Session("ann", null)
  • The key (and its @jsonName, if it has one) is accepted, so JSON written before the field was annotated still reads, but its value is discarded. The field gets zero, false, null or None, like any absent key. A field initialised in the class body is not initialised either, unless the bean is built through a constructor.
  • No serializer or deserializer is built for the field, so its type does not need to be readable or writable, and the null policy does not apply to it.
  • Like @jsonName, it works on case class parameters, vals and vars without a meta-annotation, on inherited fields and on members of traits and superclasses.

Leaving empty values out

@omitEmpty leaves an empty collection, map or array, None or "" out of the JSON, and reads a missing key as an empty value.

import io.mpjsons.omitEmpty

case class Order(id: Long, @omitEmpty tags: Set[String], @omitEmpty note: String,
                 @omitEmpty discount: Option[Int])

mpjsons.serialize(Order(1, Set.empty, "", None))     // {"id":1}
mpjsons.serialize(Order(2, Set("a"), "n", Some(5)))  // {"id":2,"tags":["a"],"note":"n","discount":{"value":5}}
mpjsons.deserialize[Order]("""{"id":1}""")           // Order(1, Set(), "", None)
  • It applies to everything that is IterableOnce (collections, maps, Option), to arrays and to String; on any other type it fails when the (de)serializer is built. With a converter registered for the field's type, any value it writes as [], {} or "" counts as empty too.
  • A missing Option key reads as None, as always. Any other missing key reads the way [] (or "") would, so converters and post-deserialization transforms registered for the field's type apply to the empty value too. Each object gets its own empty mutable collection.
  • null is not empty: it follows the null policy as before.
  • Like any absent key, a missing key keeps a value set by a no-arg constructor.

Annotations on traits and superclasses

@jsonName, @nullable, @skipSerialization and @omitEmpty can also be put on a member of a trait or superclass. They then apply to the field that implements or overrides that member, in every subclass:

sealed trait Event {
  @jsonName("at") def time: Long
  @omitEmpty def tags: Set[String]
}

case class Created(time: Long, tags: Set[String]) extends Event
case class Renamed(time: Long, name: String, tags: Set[String]) extends Event

mpjsons.serialize(Created(5, Set.empty))            // {"at":5}
mpjsons.serialize(Renamed(6, "", Set("x")))         // {"at":6,"name":"","tags":["x"]}
  • The field's own annotation wins over a supertype's; among supertypes, the nearest one in the linearization wins (in class C extends A with B, the one in B).
  • Up to 0.7.8 annotations on supertype members had no effect (see docs/upgrading.md).

Nullable fields and null handling

By default, serializing a null field throws. Mark a field with @nullable to allow it to be omitted from the output when it's null. It works directly on case-class parameters (the @(nullable @field) form shown below is still accepted):

import io.mpjsons.nullable
import scala.annotation.meta.field

case class SimpleObjectWithNulls(@(nullable @field) a: String, b: String)

mpjsons.serialize(SimpleObjectWithNulls(null, "ok"))   // OK — "a" is omitted
mpjsons.serialize(SimpleObjectWithNulls("fail", null)) // throws — "b" is not nullable

When every field of an object is @nullable and null, the object serializes to {} and round-trips back to the all-null instance:

case class PrimitiveHolder(
  @(nullable @field) a: java.lang.Long,
  @(nullable @field) b: java.lang.Integer
)

mpjsons.serialize(PrimitiveHolder(null, null))     // {}
mpjsons.deserialize[PrimitiveHolder]("{}")          // PrimitiveHolder(null, null)

The null literal

Nulls are accepted only where they are declared. A field declares that it may be null with @nullable; an Option declares absence by its type. With the default NullPolicy.Fail this holds in both directions:

  • Write: a null @nullable field is omitted; any other null (a field, a collection element, a map key or value, a tuple slot, an Either or Some payload, the top-level value) throws.
  • Read: null gives null for a @nullable reference field and None for an Option; anywhere else it is rejected with a message naming the field or type. A missing Option field also reads as None. Primitives (Int, Boolean, ...) never accept null.
import io.mpjsons.nullable

case class Profile(@nullable name: String, nickname: Option[String], city: String, age: Int)

mpjsons.deserialize[Profile]("""{"name":null,"nickname":null,"city":"Oslo","age":3}""") // Profile(null, None, "Oslo", 3)
mpjsons.deserialize[Profile]("""{"name":"Ann","city":"Oslo","age":3}""")              // nickname = None
mpjsons.deserialize[Profile]("""{"name":"Ann","city":null,"age":3}""")                // throws: city is not @nullable
mpjsons.serialize(Profile(null, None, "Oslo", 3))                                      // {"nickname":{},"city":"Oslo","age":3}
mpjsons.serialize(List("a", null))                                                    // throws

To tolerate nulls everywhere, choose another policy explicitly; it then applies to reading as well:

import io.mpjsons.NullPolicy

new MPJsons(nullPolicy = NullPolicy.Omit).serialize(Profile("Ann", None, null, 3))      // {"name":"Ann","nickname":{},"age":3}
new MPJsons(nullPolicy = NullPolicy.WriteNull).serialize(Profile("Ann", None, null, 3)) // {"name":"Ann","nickname":{},"city":null,"age":3}
new MPJsons(nullPolicy = NullPolicy.WriteNull).serialize(List("a", null))               // ["a",null]

ignoreNullFields = true is the same as NullPolicy.Omit.

Compact output

MPJsonsConfig holds opt-in format levers. The defaults reproduce the historical format, and the reader accepts both forms wherever they can be told apart, so an instance with a lever on can still read data written with it off.

import io.mpjsons.{MPJsons, MPJsonsConfig}

val compact = new MPJsons(MPJsonsConfig(escapeSlash = false, compactOptions = true))

case class Profile(name: String, nickname: Option[String], homepage: Option[String])

compact.serialize(Profile("Ann", Some("annie"), None))
// {"name":"Ann","nickname":"annie"}      (historical: {"name":"Ann","nickname":{"value":"annie"},"homepage":{}})
Lever Default Effect when changed
escapeSlash = false true / is written unescaped instead of \/. Reading is unaffected.
compactOptions = true false Some(v) is written as v; a None field is omitted and a None element is null. An Option directly inside another Option keeps the historical form so Some(None) survives. On read, the historical {} / {"value":v} is still recognised unless the element type is itself written as a JSON object (a bean, an Option, Either, a time type), in which case { is the inline element.
mapsAsObjects = true false Maps whose key type is String, a number, Boolean or Char are written as {"key":value} instead of [["key",value]]; other key types keep the pair form. A null value is treated like a null bean field (dropped under NullPolicy.Omit, "key":null under WriteNull, error under Fail). Both forms are always read.
isoTime = true false LocalDate, LocalTime, LocalDateTime and Duration are written as ISO-8601 strings ("2023-07-23", "PT1M") instead of objects. Both forms are always read.
strict = true false Reading rejects missing commas, trailing commas and content after the root value instead of tolerating them. Bare keys and whitespace are still accepted.
omitDefaults = true false Bean fields whose value equals what reading {} would produce (the no-arg constructor initialisers, or zero / false / null / None) are left out. Lossless for round trips through this library; such nulls are omitted regardless of nullPolicy.
caseClassesViaConstructor = true false Case classes are built by calling their primary constructor with the parsed values (constructor bodies such as require run; no final field is written reflectively, which future JDKs restrict). Absent keys still give zero / false / null / None. Costs some boxing per value.
maxDepth = n 0 (unlimited) Reading fails with a clear error once values (objects, collections, maps, Options, tuples) nest deeper than n, including unknown fields being skipped. Use it for untrusted input, which could otherwise end in a StackOverflowError.
maxOutputBytes = n 100 MiB Writing fails with a JsonTooLargeException once the output is larger than n bytes; see "Output size limit".

Other levers are listed in docs/roadmap.md.

Post-deserialization transform

Register a function to run on each instance of a type after deserialization — handy for cleaning up nulls left by missing JSON properties:

case class Sample(a: String, b: String)

mpjsons.postDeserializationTransform[Sample] { s =>
  if (s.b == null) s.copy(b = "") else s
}

mpjsons.deserialize[Sample]("""{a: "Hello"}""")  // Sample("Hello", "")

Custom converters

To support a type mpjsons doesn't handle natively, implement JsonTypeSerializer[T] (serialize(obj, jsonBuilder): Unit) and/or JsonTypeDeserializer[T] (deserialize(jsonIterator): T), then register them. Registration takes factory functions so your converter can resolve sub-(de)serializers from the factory.

mpjsons.registerConverter[SomeType](
  sf => new MyTypeSerializer(sf),
  df => new MyTypeDeserializer(df))

Related registration methods:

  • registerConverter[T](serializer, deserializer) — for exactly type T.
  • registerSuperclassConverter[T](serializer, deserializer) — for T and all its subtypes.
  • registerSerializerForAnySubtype[T](serializer) — serialize-only, handles T and any subtype (including generic containers like Wrapper[_]).
case class Wrapper[T](value: T)

class WrapperSerializer extends JsonTypeSerializer[Wrapper[_]] {
  override def serialize(obj: Wrapper[_], jb: StringBuilder): Unit = {
    jb.append("{\"someValue\":")
    jb.append(obj.value.toString)
    jb.append("}")
  }
}

mpjsons.registerSerializerForAnySubtype[Wrapper[_]](sf => new WrapperSerializer)
mpjsons.serialize(Wrapper(123))  // {"someValue":123}

Registrations take effect immediately, also for types that were already serialized or deserialized: every registration drops the cached (de)serializers. Static serializers and deserializers built earlier keep the behaviour they were built with, so build them after registering. Register converters before sharing an instance between threads.

Polymorphic / typed JSON

When a collection holds subtypes of a common trait, plain JSON loses the concrete type. markTypedClass[T] embeds each value's class name as a wrapper so it round-trips:

sealed trait CustomDescription
case class TextDescription(text: String) extends CustomDescription
case class NumericDescription(number: Int) extends CustomDescription

mpjsons.markTypedClass[CustomDescription]

val json = mpjsons.serialize(
  List[CustomDescription](TextDescription("Hello"), NumericDescription(14)))
// [{"TextDescription":{"text":"Hello"}},{"NumericDescription":{"number":14}}]

mpjsons.deserialize[List[CustomDescription]](json)
// List(TextDescription("Hello"), NumericDescription(14))

The type tag is added only when serializing through the supertype. Serializing the concrete type directly produces plain JSON:

trait CommonType
case class A(a: String) extends CommonType

mpjsons.markTypedClass[CommonType]

mpjsons.serialize(A("Hello"))                 // {"a":"Hello"}
mpjsons.serialize(A("Hello"): CommonType)     // {"A":{"a":"Hello"}}

markTypedSuperClass[T] behaves the same but applies to every subclass of T. It is kept for backward compatibility and generally should not be needed.

Type name aliases

By default the wrapper key is the subclass's simple name. @typeName sets a different one: shorter output, and a renamed class can keep its old name on the wire. The simple class name is still accepted when reading, so data written before the alias was added keeps loading.

import io.mpjsons.typeName

sealed trait Shape
@typeName("c") case class Circle(r: Double) extends Shape
@typeName("Square") case class Box(side: Int) extends Shape   // class renamed from Square

mpjsons.markTypedClass[Shape]
mpjsons.serialize(List[Shape](Circle(1.5), Box(2)))   // [{"c":{"r":1.5}},{"Square":{"side":2}}]

For a sealed typed class, aliases are resolved through its known subclasses, wherever they are declared. For a typed class that is not sealed (its subclasses may be in other files), Scala records no subclasses, so the first alias written or read makes mpjsons list the typed class's package on the class path (directories and jars) and read the aliases of the subclasses it finds there. This runs once per typed class and only when an alias is used; it measured 65–208 ms for packages of a few hundred to 1 300 classes. An aliased subclass in another package cannot be found: writing it fails with an explanation. Two subclasses that would write the same name are rejected.

Performance: static (de)serializers

serialize/deserialize look the (de)serializer up per call (cached after the first use). On hot paths you can bind a reusable, type-specialized instance once:

val ser = mpjsons.buildStaticSerializer[User]
val des = mpjsons.buildStaticDeserializer[User]

val json = ser.serialize(user)
val back = des.deserialize(json)

StaticSerializer also exposes serializeToStringBuilder(obj): StringBuilder, which avoids the char-array copy of .toString() when a StringBuilder can be consumed directly.

Byte output

When the JSON goes to an HTTP response, a file or another byte sink, serializeToBytes writes UTF-8 directly instead of building a String and encoding it. The bytes are exactly serialize(obj).getBytes(UTF_8):

val bytes: Array[Byte] = mpjsons.serializeToBytes(user)   // also (obj, Type) and (obj, typeName)
val ser = mpjsons.buildStaticSerializer[User]
ser.serializeToBytes(user)

Custom converters keep writing into a StringBuilder; their output is encoded to UTF-8.

With Pekko HTTP, wrap the bytes without another copy and send them as application/json:

complete(HttpEntity.Strict(ContentTypes.`application/json`, ByteString.fromArrayUnsafe(ser.serializeToBytes(user))))

complete(json: String) would be sent as text/plain, and HttpEntity(contentType, bytes) copies the array.

serializeToByteBuffer returns the same bytes without the final copy that serializeToBytes makes: a ByteBuffer wrapping the array they were written into (usually larger than the output; remaining is the output's length):

val buffer: java.nio.ByteBuffer = ser.serializeToByteBuffer(user)
complete(HttpEntity.Strict(ContentTypes.`application/json`, ByteString.fromArrayUnsafe(buffer.array, 0, buffer.limit)))

ByteString.fromByteBuffer(buffer) would copy the bytes again, which gains nothing over serializeToBytes.

As a rule, keep this (or serializeToBytes) for ordinary responses and stream only those that can be large (exports, reports, unpaginated lists; roughly 1 MB and more). Streaming holds a blocking thread per response for as long as the client takes to read it, a failure part-way can only break the connection instead of becoming an error response, and the response is sent chunked, without Content-Length.

Streaming output

serializeToStream writes the same bytes to an OutputStream while they are produced, in chunks of 32 KiB, so a large response does not have to be held in memory: memory use stays at about one chunk, or the longest single string value (or one custom converter's output) when that is larger. It flushes the stream at the end and does not close it. Writing to the stream blocks, so run it on a thread that may block.

mpjsons.serializeToStream(report, outputStream)   // also (obj, Type, stream), (obj, typeName, stream)
ser.serializeToStream(report, outputStream)       // StaticSerializer

The chunk size is MPJsonsConfig.streamChunkBytes (32 KiB by default), or the chunkBytes argument for one call, which comes after maxBytes. One write to the stream can be up to about twice the chunk size. Larger chunks write to the stream less often, which is faster through Pekko's asOutputStream (each chunk costs a hand-off between threads), but hold more memory per response in progress:

val mpjsons = new MPJsons(MPJsonsConfig(streamChunkBytes = 16 * 1024))   // every stream of this instance
ser.serializeToStream(report, outputStream, 800 * 1024 * 1024, 256 * 1024)  // this call: maxBytes, chunkBytes

Streaming a 10.8 MB document through asOutputStream (measured once on one machine) took 173 ms in 1 KiB chunks, 48 ms in 8 KiB, 34 ms in 32 KiB and 30 ms in 256 KiB; mpjsons itself took 24 ms at every size. 32 KiB suits most responses; 128–256 KiB pays off for multi-megabyte exports with few of them at a time. Above 256 KiB the gain is small, and with G1 a buffer of half a heap region or more (regions are 1–32 MB depending on the heap) is a slower "humongous" allocation.

If serialization fails part-way (an exception from a serializer, or the output limit), the chunks already written stay written. With Pekko HTTP the status line and headers are sent before the body is complete, so a failure can no longer change the status; fail the response stream instead of ending it, so the client sees a broken response rather than a short JSON:

import org.apache.pekko.NotUsed
import org.apache.pekko.stream.scaladsl.{Source, StreamConverters}
import org.apache.pekko.util.ByteString
import scala.concurrent.{ExecutionContext, Future, Promise}

/** `blocking`: a dispatcher for blocking work, e.g. system.dispatchers.lookup("pekko.actor.default-blocking-io-dispatcher"). */
def jsonSource[T](ser: StaticSerializer[T], value: T)(implicit blocking: ExecutionContext): Source[ByteString, NotUsed] = {
  val end = Promise[Source[ByteString, NotUsed]]()
  StreamConverters.asOutputStream()
    .mapMaterializedValue { out =>
      Future(ser.serializeToStream(value, out)).onComplete { result =>
        out.close()
        end.tryComplete(result.map(_ => Source.empty)) // a failure fails the stream instead of ending it
      }
      NotUsed
    }
    .concat(Source.futureSource(end.future))
}

complete(HttpEntity(ContentTypes.`application/json`, jsonSource(ser, report)))  // sent chunked

For small responses serializeToBytes is faster: the stream costs a thread switch and chunked transfer per response.

Output size limit

The JDK cannot build a String of 2^30 (about a billion) or more characters when any of them is outside Latin-1, e.g. a Polish letter: the attempt ends in OutOfMemoryError: UTF16 String size is ... after using several gigabytes of memory. mpjsons stops earlier, at MPJsonsConfig.maxOutputBytes (100 MiB by default). Larger output fails with an io.mpjsons.JsonTooLargeException (an IllegalArgumentException) naming the type and the limit. Every serialize method also takes a limit in bytes for one call:

val mpjsons = new MPJsons(MPJsonsConfig(maxOutputBytes = 20 * 1024 * 1024))
mpjsons.serialize(event)                                     // up to 20 MiB
mpjsons.serialize(bigReport, maxBytes = 800 * 1024 * 1024)  // this call only
mpjsons.serializeToBytes(bigReport, maxBytes = 800 * 1024 * 1024)
mpjsons.serializeToStream(bigReport, outputStream, maxBytes = 800 * 1024 * 1024)
mpjsons.buildStaticSerializer[Report].serialize(bigReport, 800 * 1024 * 1024)

For serialize the limit counts characters, which is never more than the UTF-8 bytes they encode to. Byte output stops as soon as its buffer would grow past the limit. String output is checked after each object, every 1024 elements of a collection or array, after each map entry and before a long string is appended, so it stops soon after crossing the limit, and the finished output is checked exactly. Keep the limit well below a billion so that this margin cannot reach the JDK's limit.

Error handling

Deserialization failures (malformed JSON, unknown fields in strict mode, type mismatches) are wrapped in io.mpjsons.impl.JsonInnerException. The message points at the exact location in the input where parsing failed:

import io.mpjsons.impl.JsonInnerException

try {
  mpjsons.deserialize[User]("""{"firstName":"Bob" """) // truncated
} catch {
  case e: JsonInnerException => println(e.getMessage)
}

Upgrading from 0.6.51 to 0.7.0

The default wire format is unchanged except for key order (declaration order, superclass fields first) and a handful of fixes that change output or accepted input (inherited fields, generic beans, null, absent Option fields reading as None, non-finite doubles). The full list, including API and dependency changes, is in docs/upgrading.md.

Limitations & notes

  • Traits and abstract classes need markTypedClass or a converter. Serializing or deserializing a value through such a type otherwise fails when first used, because the concrete class would be lost.
  • Map form is non-standard. Regular maps serialize as an array of [key, value] pairs (see Maps); use SimpleMap* converters for the {"k":"v"} object form.
  • Sorted collections throw. SortedSet, TreeSet, SortedMap, TreeMap, and ListBuffer are intentionally unsupported and throw.
  • Singleton objects serialize to {} and deserialize to the singleton instance when the declared type is Foo.type; polymorphic use (an object behind a trait) still needs markTypedClass.
  • Constructor-less deserialization uses sun.misc.Unsafe. When a type has no no-arg constructor, instances are allocated without running any constructor — so constructor-side defaults and validation are bypassed, and availability depends on the JVM.
  • Not thread-safe for cold types. Internal caches are unsynchronized var Maps. A single MPJsons instance is not safe for concurrent first-time use of new types; once all types are warmed up it is effectively read-only. Warm up types up front if you serialize concurrently.

Building & testing

sbt compile
sbt test                                       # all specs
sbt "testOnly io.mpjsons.BeanSerializationSpec" # a single spec

Specs live under src/test/scala/io/mpjsons/, one per feature area (collections, maps, tuples, Option/Either, time, custom converters, malformed JSON, and more) and are the authoritative reference for supported behavior.

License

Released under the MIT License.

About

Marcin Pieciukiewicz's JSON library for Scala.

Resources

Readme
Apache-2.0 license
Activity

Stars

0 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.