path: root/doc/articles/json_and_go.html

<!--{
"Title": "JSON and Go",
"Template": true
}-->

<p>
JSON (JavaScript Object Notation) is a simple data interchange format.
Syntactically it resembles the objects and lists of JavaScript. It is most
commonly used for communication between web back-ends and JavaScript programs
running in the browser, but it is used in many other places, too. Its home page,
<a href="http://json.org">json.org</a>, provides a wonderfully clear and concise
definition of the standard.
</p>

<p>
With the <a href="/pkg/encoding/json/">json package</a> it's a snap to read and
write JSON data from your Go programs.
</p>

<p>
<b>Encoding</b>
</p>

<p>
To encode JSON data we use the
<a href="/pkg/encoding/json/#Marshal"><code>Marshal</code></a> function.
</p>

<pre>
func Marshal(v interface{}) ([]byte, error)
</pre>

<p>
Given the Go data structure, <code>Message</code>,
</p>

{{code "/doc/progs/json1.go" `/type Message/` `/STOP/`}}

<p>
and an instance of <code>Message</code>
</p>

{{code "/doc/progs/json1.go" `/m :=/`}}

<p>
we can marshal a JSON-encoded version of <code>m</code> using <code>json.Marshal</code>:
</p>

{{code "/doc/progs/json1.go" `/b, err :=/`}}

<p>
If all is well, <code>err</code> will be <code>nil</code> and <code>b</code>
will be a <code>[]byte</code> containing this JSON data:
</p>

<pre>
b == []byte(`{"Name":"Alice","Body":"Hello","Time":1294706395881547000}`)
</pre>

<p>
Only data structures that can be represented as valid JSON will be encoded:
</p>

<ul>
<li>
JSON objects only support strings as keys; to encode a Go map type it must be
of the form <code>map[string]T</code> (where <code>T</code> is any Go type
supported by the json package).
</li>
<li>
Channel, complex, and function types cannot be encoded.
</li>
<li>
Cyclic data structures are not supported; they will cause <code>Marshal</code>
to go into an infinite loop.
</li>
<li>
Pointers will be encoded as the values they point to (or 'null' if the pointer
is <code>nil</code>).
</li>
</ul>

<p>
The json package only accesses the exported fields of struct types (those that
begin with an uppercase letter). Therefore only the exported fields of a struct
will be present in the JSON output.
</p>

<p>
<b>Decoding</b>
</p>

<p>
To decode JSON data we use the
<a href="/pkg/encoding/json/#Unmarshal"><code>Unmarshal</code></a> function.
</p>

<pre>
func Unmarshal(data []byte, v interface{}) error
</pre>

<p>
We must first create a place where the decoded data will be stored
</p>

{{code "/doc/progs/json1.go" `/var m Message/`}}

<p>
and call <code>json.Unmarshal</code>, passing it a <code>[]byte</code> of JSON
data and a pointer to <code>m</code>
</p>

{{code "/doc/progs/json1.go" `/err := json.Unmarshal/`}}

<p>
If <code>b</code> contains valid JSON that fits in <code>m</code>, after the
call <code>err</code> will be <code>nil</code> and the data from <code>b</code>
will have been stored in the struct <code>m</code>, as if by an assignment
like:
</p>

{{code "/doc/progs/json1.go" `/m = Message/` `/STOP/`}}

<p>
How does <code>Unmarshal</code> identify the fields in which to store the
decoded data? For a given JSON key <code>"Foo"</code>, <code>Unmarshal</code>
will look through the destination struct's fields to find (in order of
preference):
</p>

<ul>
<li>
An exported field with a tag of <code>`json:"Foo"`</code> (see the
<a href="/ref/spec#Struct_types">Go spec</a> for more on struct tags),
</li>
<li>
An exported field named <code>"Foo"</code>, or
</li>
<li>
An exported field named <code>"FOO"</code> or <code>"FoO"</code> or some other
case-insensitive match of <code>"Foo"</code>.
</li>
</ul>

<p>
What happens when the structure of the JSON data doesn't exactly match the Go
type?
</p>

{{code "/doc/progs/json1.go" `/"Food":"Pickle"/` `/STOP/`}}

<p>
<code>Unmarshal</code> will decode only the fields that it can find in the
destination type.  In this case, only the <code>Name</code> field of m will be
populated, and the <code>Food</code> field will be ignored. This behavior is
particularly useful when you wish to pick only a few specific fields out of a
large JSON blob. It also means that any unexported fields in the destination
struct will be unaffected by <code>Unmarshal</code>.
</p>

<p>
But what if you don't know the structure of your JSON data beforehand?
</p>

<p>
<b>Generic JSON with <code>interface{}</code></b>
</p>

<p>
The <code>interface{}</code> (empty interface) type describes an interface with
zero methods.  Every Go type implements at least zero methods and therefore
satisfies the empty interface.
</p>

<p>
The empty interface serves as a general container type:
</p>

{{code "/doc/progs/json2.go" `/var i interface{}/` `/STOP/`}}

<p>
A type assertion accesses the underlying concrete type:
</p>

{{code "/doc/progs/json2.go" `/r := i/` `/STOP/`}}

<p>
Or, if the underlying type is unknown, a type switch determines the type:
</p>

{{code "/doc/progs/json2.go" `/switch v/` `/STOP/`}}

<p>
The json package uses <code>map[string]interface{}</code> and
<code>[]interface{}</code> values to store arbitrary JSON objects and arrays;
it will happily unmarshal any valid JSON blob into a plain
<code>interface{}</code> value.  The default concrete Go types are:
</p>

<ul>
<li>
<code>bool</code> for JSON booleans,
</li>
<li>
<code>float64</code> for JSON numbers,
</li>
<li>
<code>string</code> for JSON strings, and
</li>
<li>
<code>nil</code> for JSON null.
</li>
</ul>

<p>
<b>Decoding arbitrary data</b>
</p>

<p>
Consider this JSON data, stored in the variable <code>b</code>:
</p>

{{code "/doc/progs/json3.go" `/b :=/`}}

<p>
Without knowing this data's structure, we can decode it into an
<code>interface{}</code> value with <code>Unmarshal</code>:
</p>

{{code "/doc/progs/json3.go" `/var f interface/` `/STOP/`}}

<p>
At this point the Go value in <code>f</code> would be a map whose keys are
strings and whose values are themselves stored as empty interface values:
</p>

{{code "/doc/progs/json3.go" `/f = map/` `/STOP/`}}

<p>
To access this data we can use a type assertion to access <code>f</code>'s
underlying <code>map[string]interface{}</code>:
</p>

{{code "/doc/progs/json3.go" `/m := f/`}}

<p>
We can then iterate through the map with a range statement and use a type switch
to access its values as their concrete types:
</p>

{{code "/doc/progs/json3.go" `/for k, v/` `/STOP/`}}

<p>
In this way you can work with unknown JSON data while still enjoying the
benefits of type safety.
</p>

<p>
<b>Reference Types</b>
</p>

<p>
Let's define a Go type to contain the data from the previous example:
</p>

{{code "/doc/progs/json4.go" `/type FamilyMember/` `/STOP/`}}

{{code "/doc/progs/json4.go" `/var m FamilyMember/` `/STOP/`}}

<p>
Unmarshaling that data into a <code>FamilyMember</code> value works as
expected, but if we look closely we can see a remarkable thing has happened.
With the var statement we allocated a <code>FamilyMember</code> struct, and
then provided a pointer to that value to <code>Unmarshal</code>, but at that
time the <code>Parents</code> field was a <code>nil</code> slice value. To
populate the <code>Parents</code> field, <code>Unmarshal</code> allocated a new
slice behind the scenes. This is typical of how <code>Unmarshal</code> works
with the supported reference types (pointers, slices, and maps).
</p>

<p>
Consider unmarshaling into this data structure:
</p>

<pre>
type Foo struct {
    Bar *Bar
}
</pre>

<p>
If there were a <code>Bar</code> field in the JSON object,
<code>Unmarshal</code> would allocate a new <code>Bar</code> and populate it.
If not, <code>Bar</code> would be left as a <code>nil</code> pointer.
</p>

<p>
From this a useful pattern arises: if you have an application that receives a
few distinct message types, you might define "receiver" structure like
</p>

<pre>
type IncomingMessage struct {
    Cmd *Command
    Msg *Message
}
</pre>

<p>
and the sending party can populate the <code>Cmd</code> field and/or the
<code>Msg</code> field of the top-level JSON object, depending on the type of
message they want to communicate. <code>Unmarshal</code>, when decoding the
JSON into an <code>IncomingMessage</code> struct, will only allocate the data
structures present in the JSON data. To know which messages to process, the
programmer need simply test that either <code>Cmd</code> or <code>Msg</code> is
not <code>nil</code>.
</p>

<p>
<b>Streaming Encoders and Decoders</b>
</p>

<p>
The json package provides <code>Decoder</code> and <code>Encoder</code> types
to support the common operation of reading and writing streams of JSON data.
The <code>NewDecoder</code> and <code>NewEncoder</code> functions wrap the
<a href="/pkg/io/#Reader"><code>io.Reader</code></a> and
<a href="/pkg/io/#Writer"><code>io.Writer</code></a> interface types.
</p>

<pre>
func NewDecoder(r io.Reader) *Decoder
func NewEncoder(w io.Writer) *Encoder
</pre>

<p>
Here's an example program that reads a series of JSON objects from standard
input, removes all but the <code>Name</code> field from each object, and then
writes the objects to standard output:
</p>

{{code "/doc/progs/json5.go" `/package main/` `$`}}

<p>
Due to the ubiquity of Readers and Writers, these <code>Encoder</code> and
<code>Decoder</code> types can be used in a broad range of scenarios, such as
reading and writing to HTTP connections, WebSockets, or files.
</p>

<p>
<b>References</b>
</p>

<p>
For more information see the <a href="/pkg/encoding/json/">json package documentation</a>. For an example usage of
json see the source files of the <a href="/pkg/net/rpc/jsonrpc/">jsonrpc package</a>.
</p>