Ten years of sharpening a sword go 1.18 generics

what are generics


Generic programming is a style or paradigm of programming languages. Generics allow programmers writing code in strongly typed programming languages ​​to use types that are specified later, specifying these types as parameters at instantiation time. Various programming languages ​​and their compilers and runtime environments support generics differently. Java and C# call it generics, ML, Scala and Haskell call it parametric polymorphism; C++ and D call it a template. The widely influential 1994 edition of Design Patterns called it parameterized types.
Why do you need generics
Considering such a requirement, implement a function that accepts two ints as input parameters and returns the smaller of the two. The requirements are very simple, we can write the following code without thinking:
func Min(a,b int) int {
if a < b {
return a
}
return b
}

It looks beautiful, but this function has limitations. The input parameters can only be of int type. If the requirements are expanded, it is necessary to support the judgment of two float64 input parameters and return the smaller of the two.
As we all know, go is a strongly typed language, and unlike c, there are implicit type conversions in arithmetic expressions (such as implicit int to bool, float to int), so the above function cannot meet the demand scenario. , but to support this extended requirement is also very simple, change it to the following code and then use MinFloat64:
func Min(a,b int) int {
if a < b {
return a
}
return b
}
func MinFloat64(a,b float64) float64 {
if a < b {
return a
}
return b
}

However, if the requirement is extended, it needs to support two int64 types. The same is also very simple, as follows:
func Min(a,b int) int {
if a < b {
return a
}
return b
}
func MinFloat64(a,b float64) float64 {
if a < b {
return a
}
return b
}
func MinInt64(a,b int64) int64 {
if a < b {
return a
}
return b
}

But if the demand is expanded again...then we keep going on and on, and finally it becomes like the picture below (ps: go is one sublime away from generics...)

I don’t know if you have noticed that once the requirements are expanded, we all need to make some changes and repeat things all the time, and by looking at the function prototype, we find that only the type declaration is inconsistent here, and of course the function name is also inconsistent, because golang It also does not support function overloading. If golang supports function overloading, the only thing that is inconsistent here is the type (ps: function overloading is actually an implementation of generics, which is passed at compile time. By adding the type parameter information to the function symbol, the function of the same name can be called during encoding, but there will be no ambiguity at runtime because of the type information).

So is there a way to reduce our repetitive workload? After the requirements are expanded, support can be provided without changing the original code, that is, to improve the reusability of the code, and this is the mission of generics.
before go1.18 Generics
Before generics, how did developers implement "generics".


copy & paste


This is the easiest way we can think of, and it is also the way we introduced in the previous article. It seems to be a very stupid way, but combined with the actual situation, in most cases, you may only need two or three types of implementations. Premature Going to optimization may bring more problems. There is a sentence in go proverbs that fits this scenario very well.

"A little copying is better than a little dependency."

Advantages: No additional dependencies are required, and the code logic is simple.
Disadvantages: The code will be a bit bloated and there is a lack of flexibility.


interface


It is more in line with the idea of ​​OOP, and interface-oriented programming is easy to think of this way, but like the above-mentioned two-digit min scenario, it cannot be satisfied by interface, and the applicable scenarios are relatively simple, consider the following interface.
type Inputer interface {
Input() string
}

For the Inputer interface, we can define multiple implementations, such as
type MouseInput struct{}

func (MouseInput) Input() string {
return "MouseInput"
}

type KeyboardInput struct{}
func (KeyboardInput) Input() string {
return "KeyboardInput"
}

In this way, when we call, we can define the same interface with different types, and call the same function through the interface. However, in essence, interface and generic are two design ideas, and the application scenarios are not the same. Here is just a common example.
Advantages: No additional dependencies are required, and the code logic is simple.
Disadvantages: The code will be a bit bloated, and the application scenario is relatively single.


reflect


Reflect (reflection) dynamically obtains types at runtime, and golang runtime stores all types used. For user-level golang, it provides a very powerful reflection package, which sacrifices performance, but provides more convenience and helps programs You can use some dynamic features in static languages. In essence, reflect and generic are two completely different design ideas. Reflection plays a role at runtime, while generics play a role at compile time. Runtime does not need to perceive generics. The existence of , like the gorm framework, uses a lot of reflection. The reflect package has a built-in implementation of DeepEqual, which is used to determine whether two input parameters are equal.
func DeepEqual(x, y any) bool {
if x == nil || y == nil {
return x == y
}
v1 := ValueOf(x)
v2 := ValueOf(y)
if v1.Type() != v2.Type() {
return false
}
return deepValueEqual(v1, v2, make(map[visit]bool))
}

Advantages: The code is simple and easy to use.
Disadvantages: high runtime overhead, unsafe, no compile-time type guarantees.
(ps: Those who have used reflection have basically encountered panic, the type guarantee at runtime, there are a lot of type checks in the reflect package, and those that do not conform to panic directly. I have doubts here, reflect package and map/slice are not very In the same way, it is more user-friendly, why not use error, but use panic. The guess is that the go team thinks that the type mismatch in static language is a very serious scenario?)


code generator


Code generation, the one that everyone has been exposed to more may be the code generation of thrift/grpc, which converts idl into the corresponding language source code. The concept of the code generator here will be different. The concept may be similar to the previous php/jsp. Write a general template, preset some variables in the template, and then use tools to fill in the preset variables to generate The final language code (ps: it seems to be similar to generics, hahaha), go also introduced the go generator tool in 1.5, which is generally used in conjunction with the text/template package, and there are more popular third parties in the go code generator Tool: github.com/cheekybits/… generator to write the Min of two numbers, it will be the following style:
package main

import "github.com/cheekybits/genny/generic"

//go:generate genny -in=$GOFILE -out=gen-$GOFILE gen "T=int,float32,float64"
type T generic.Type

func MinT(a, b T) T {
if a < b {
return a
}
return b
}

Executing the go generator will generate the following code:
// This file was automatically generated by genny.
// Any changes will be lost if this file is regenerated.
// see https://github.com/cheekybits/genny

package main

func MinInt(a, b int) int {
if a < b {
return a
}
return b
}

func MinFloat32(a, b float32) float32 {
if a < b {
return a
}
return b
}

func MinFloat64(a, b float64) float64 {
if a < b {
return a
}
return b
}

Advantages: The code is relatively clean, because it is generated before use, and it can also take advantage of the ability of static checking, which is safe and has no runtime overhead.
Disadvantages: It is necessary to write the template code in a targeted manner, and then use the tool to generate the final code before it can be used in the project, and rely on third-party build tools, because multiple types of source code generation are involved, the code in the project will increase, The resulting binary will also be larger.
go 1.18 Generics
The journey of go generics is also very tortuous...

It has been designed since 2010. The Contracts (contract) scheme proposed in the development process was once considered to be a generic implementation, but in 2019, it was abandoned because the design was too complicated, and it was not confirmed until 2021. The final basic solution was implemented, and the beta version was implemented in golang 1.17 in August 2021, and in golang 1 in January 2022.18 has been installed in the real sense of ten years of sharpening a sword (ps: Ian Lance Taylor is too good).
generic type
There is a number type in json. When encountering the interface{} type in golang's encoding/json library, float64 is used to parse the number type of json by default, which will lead to loss of precision in the face of large integers, but in fact The Number type should correspond to multiple types in golang, including int32, int64, float32 and float64, etc. If according to the grammar of golang, we can identify the Number type in the generic type.
type Number[T int32|int64|float32|float64] T

But unfortunately. . . At present, golang does not support this way of writing, and the following error will be reported when compiling:
cannot use a type parameter as RHS in type declaration
//RHS:right hand side (on the right side of the operator)

The meaning of the error is that it is not yet supported to use type parameters alone as generic types. It needs to be used in combination with types such as struct, slice and map. For a discussion on this issue, see: github.com/golang/go/i… Lance Taylor The big guy makes a reply: It means that this is a known problem of go1.18 generics, and it will probably be tried in go 1.19.
We try to define a generic Number slice type and instantiate it using:
package main

type Numbers[T int32 | int64 | float32 | float64] []T

func main() {
var a = Numbers[int32]{1, 2, 3}
println(a)
}


T is a type parameter. This keyword is not fixed. We can have any name. Its function is to occupy a place. It indicates that there is a type here, but the specific type depends on the type that follows. constraint.



int32|int64|float32|float64 This string of type lists separated by "or identifier|" is a type constraint, which constrains the actual type type of T, and we also call this type list a type parameter list ( type parameter list)



The type defined here is Numbers[T], which is called a generic type, and a generic type will have formal parameters when it is defined.



And the []T defined here is called the defined type (defined type)



Numbers[int32] in the main function is to instantiate the generic type. The generic type can only be used after instantiation. The int32 here is the specific instantiation type, which must be defined in the type constraint. types, called type arguments

This is actually instantiating a slice of int32 with a length of 3 and elements 1, 2, and 3 in sequence. Similarly, we can also define it as follows, and float32 is also in our type parameter list.
var b = Numbers[float32]{1.1, 2.1, 3.1}

The above is a generic type with only one parameter, let's look at several complex generic types.

Multiple type parameters

type KV[K int32 | float32,V int8|bool] map[K]V// (defines of multiple type parameters are separated by commas)
var b = KV[int32, bool]{10: true}

We have defined the generic type KV[K,V] above, K and V are type parameters, the type constraint of K is int32|float32, the type constraint of V is int8|bool, K int32 | float32, V int8|bool It is the type parameter list of the KV type, and KV[int32, bool] is the instantiation of the generic type, where int32 is the actual parameter of K, and bool is the actual parameter of V.

nested parameters

type User[T int32 | string, TS []T | []string] struct {
Id T
Emails TS
}
var c = User[int32, []string]{
Id: 10,
Emails: []string{"123@qq.com", "456@gmail.com"},
}

This type looks more complicated, but golang has a limitation: any defined formal parameter needs to have one-to-one corresponding actual parameters in order when used. Above we have defined the generic type struct{Id T Email TS}, T and TS are type parameters, the type constraint of T is int32|string, and the type constraint of TS is []T|[]string, that is, We use the pre-defined T parameter in the type constraint of the TS parameter defined here, and this syntax is also supported by golang.

Nesting of formal parameter conduction

type Ints[T int32|int64] []T
type Int32s[T int32] Ints[T]

Here we define the Ints type, the formal parameter is int32|int64, and based on the Ints type, the Int32s type is defined, which is the code in our second line, which may seem confusing at first, but take it apart:
Int32s[T] is a generic type, T is a type parameter, the type constraint of T is int32, and Ints[T] is the defined type here, and the defined type here is a generic type, and instantiating this generic type The way of type is to use the actual parameter T to instantiate. Note that T is the formal parameter of Int32s here, and it is indeed the actual parameter of Ints.
generic function
Only generic types cannot play the real role of generics. The most powerful role of generics is to use them in combination with functions. Going back to our initial example, take the min of the two numbers. In the case of generics, We can write code like this:
package main


func main() {
println(Min[int32](10, 20))
println(Min[float32](10, 20))
}

func Min[T int | int32 | int64 | float32 | float64](a, b T) T {
if a < b {
return a
}
return b
}

We have defined the Min generic function above, including the generic T type, with corresponding type constraints. In the actual call, we use int32/float32 to instantiate the formal parameters to call different types of generic functions.
The above will also be inconvenient to use. We also need to explicitly specify the type when calling to use the generic function. Golang supports auto type inference for this situation, which can simplify our Writing, we can call the Min function in the following way.
Min(10, 20)//Integer literals are deduced as int in golang, so the actual instantiated function here is Min[int]
Min(10.0, 20.0)//The floating-point number literal is deduced as float64, so the instantiation function called here is Min[float64]

With generic functions, some common operations, such as set operations, intersection/union/complement/difference sets can also be written very simply. In the past, third-party libs were generally implemented by reflection, such as: github .com/thoas/go-fu…
Combining generic types and generic functions is to use generic receivers to construct advanced collection data structures, such as stacks that are more common in other languages.
package main

import (
"fmt"
)

type Stack[T interface{}] struct {
Elems []T
}

func (s *Stack[T]) Push(elem T) {
s.Elems = append(s.Elems, elem)
}

func (s *Stack[T]) Pop() (T, bool) {
var elem T
if len(s.Elems) == 0 {
return elem, false
}
elem = s.Elems[len(s.Elems)-1]
s.Elems = s.Elems[:len(s.Elems)-1]
return elem, true
}

func main() {
s := Stack[int]{}
s.Push(10)
s.Push(20)
s.Push(30)
fmt.Println(s)
fmt.Println(s.Pop())
fmt.Println(s)
}
// output:
//{[10 20 30]}
//30 true
//{[10 20]}

We have defined the generic type Stack[T] above. We use an empty interface: interface{} as a generic constraint. The meaning of an empty interface is that it does not limit the specific type, that is, it can be instantiated with all types. Pop and Push operations are implemented. With generics, advanced data structures such as queues, priority queues, and Sets that are common in other languages ​​can also be implemented relatively simply (like some previous third-party libs are generally implemented by reflection ).
The point here is that generics do not support direct use of the type assertions we used before.
func (s *Stack[T]) Push(elem T) {
switch elem.(type) {
case int:
fmt.Println("int push")
case bool:
fmt.Println("bool push")
}
s.Elems = append(s.Elems, elem)
}

//cannot use type switch on type parameter value elem (variable of type T constrained by any)

If you want to get the actual type of a generic type, you can do it by converting to interface{} (of course, you can also use reflection).
func (s *Stack[T]) Push(elem T) {
var a interface{}
a = elem
switch a.(type) {
case int:
fmt.Println("int push")
case bool:
fmt.Println("bool push")
}
s.Elems = append(s.Elems, elem)
}

interface
There are two types of built-in types in golang: basic types and composite types.
Basic data types include: Boolean, Integer, Float, Complex, Character, String, and Error.
Composite data types include: pointers, arrays, slices, dictionaries, channels, structures, and interfaces.
By combining basic types and composite types, we can define a lot of generic types, but a large number of types will lead to very long type constraints. Take number as an example:
type Numbers[T int|int8|int16|int32|int64|float32|float64] []T

define type constraints
golang supports the use of interface to predefine type constraints, so that we can reuse existing type constraints when using them, as follows:
type Number interface {
int | int8 | int16 | int32 | int64 | float32 | float64
}

type Numbers[T Number] []T

Built-in types can be freely combined to form generics. Similarly, interfaces can also be combined with interfaces, and interfaces can also be combined with built-in types to form generics.
type Int interface {
int | int8 | int16 | int32 | int64
}

type UInt interface {
uint | uint8 | uint16 | uint32 | uint64
}

type IntAndUInt interface {
Int | UInt
}

type IntAndString interface {Int | string
}

The same golang also has two built-in interfaces for our convenience, any and comparable.
any
// any is an alias for interface{} and is equivalent to interface{} in all ways.
type any = interface{}

any is actually very simple. In fact, it is an alias for an empty interface (interface{}). We have also used empty interfaces above. An empty interface can be used as any type. Using any can be more convenient for us to use, and from the semantics From the above, the semantics of any will be clearer than the semantics of interface{}.
comparable
// comparable is an interface that is implemented by all comparable types
// (booleans, numbers, strings, pointers, channels, arrays of comparable types,
// structs whose fields are all comparable types).
// The comparable interface may only be used as a type parameter constraint,
// not as the type of a variable.
type comparable interface{ comparable }

Golang has built-in comparison types, which are a combination of these built-in types mentioned in the above comments. It is also for convenience. It is worth mentioning that comparable supports == and != operations, but comparisons like > and < are not Supported, we need to implement this ordered type ourselves.
func Min[T comparable](a, b T) T {
if a < b {
return b
}
return a
}
//invalid operation: a < b (type parameter T is not comparable with <)

Of course, we can implement a comparison type ourselves:
type Signed interface {
~int | ~int8 | ~int16 | ~int32 | ~int64
}

type Unsigned interface {
~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr
}

type Integer interface {
Signed | Unsigned
}

type Float interface {
~float32 | ~float64
}

// Ordered is a constraint that permits any ordered type: any type
// that supports the operators < <= >= >.
// If future releases of Go add new ordered types,
// this constraint will be modified to include them.
type Ordered interface {
Integer | Float | ~string
}

And this is the implementation of the official golang expansion package: pkg.go.dev/golang.org/…
interface collection operations

union

What we have been using above has always been the union operation, that is, multiple types separated by vertical bars:
type Float interface {
float32 | float64
}

The above Float type constraints support the instantiation of float32/float64.

intersection

The same interface also supports the intersection operation, and the types are written to multiple lines. The final type constraint defined by the interface is the intersection of these line constraints:
type Float interface {
float32 | float64
}
type Float32 interface {
Float
float64
}

Here we define Float32 as the intersection of Float and float64, and Float is float32|float64, so Float32 actually only defines a generic constraint of float32 (belongs to yes).

empty set

Through the empty intersection we can define empty interface constraints, such as
type Null interface {
float32
int32
}

The Null we defined above is the intersection of float32 and int32. The intersection of these two types is empty, so the final Null defined is an empty type constraint. The compiler will not prevent us from using it in this way, but in fact there is nothing significance.
~ symbol
In the implementation of the Ordered type constraint above, we saw the ~ operator. This operator means that when instantiating a generic type, not only can the corresponding actual parameter type be used directly, but if the underlying type of the actual parameter is in the type It can also be used in constraints. It may be more abstract. Let’s take a look at a piece of code.
package main

type MyInt int

type Ints[T int | int32] []T

func main() {
a := Ints[int]{10, 20} //correct
b := Ints[MyInt]{10, 20}//error
println(a)
println(b)
}
//MyInt does not implement int|int32 (possibly missing ~ for int in constraint int|int32)

So in order to support this newly defined type but the convenience of the underlying type, golang adds a new ~ character, which means that if the underlying type matches, the generic instantiation can be performed normally. So it can be written as follows:
type Ints[T ~int | ~int32] []T

interface changes
Go reuses the interface keyword to define generic constraints, so the definition of interface will naturally change. Before go1.18, the definition of interface was: go.dev/doc/go1.17_…
An interface type specifies a method set called its interface

The definition of interface is method set (method set), which is indeed the case. Before go1.18, interface was a collection of methods.
type ReadWriter interface {
Read(p []byte) (n int, err error)
Write(p []byte) (n int, err error)
}

The above ReadWriter type defines the two methods of Read and Write, but we may look at the problem in reverse. There are multiple types that implement the ReadWrite interface, then we can regard ReadWrite as a collection of multiple types, and this Each type in the type set implements the two methods defined by ReadWrite. Here we take the empty interface interface{} above as an example. Because each type implements the empty interface, the empty interface can be used to identify all A collection of types, which is the any keyword we introduced earlier.
So combined with the type set that we introduced above to use interface to define generic constraints, in go1.18, the definition of interface was replaced with: go.dev/ref/spec#In…
An interface type defines a type set.

The interface is a type set (type set), and the definition of the interface has changed from a method set to a type set. A variable of an interface type can store a value of any type in the set of interface types. For the compatibility promised by golang, the interface is divided into two types, namely

basic interface



general interface

two interfaces
basic interface
If there are only methods and no types in the interface definition (it is also the definition of the interface before go1.18, and the usage is basically the same), then this interface is a basic interface.

The basic interface can define variables, such as the most commonly used error, which is consistent with the definition before go1.18

// The error built-in interface type is the conventional interface for
// representing an error condition, with the nil value representing no error.
type error interface {
Error() string
}

var err error


Base interfaces can also act as type constraints, e.g.

package main

import (
"bytes"
"io"
"strings"
)

type ReadOrWriters[T io.Reader | io.Writer] []T

func main() {
rs := ReadOrWriters[io.Reader]{bytes.NewReader([]byte{}), bytes.NewReader([]byte{})}
ws := ReadOrWriters[io.Writer]{&strings.Builder{}, &strings.Builder{}}
}

General interface
As long as the interface contains type constraints (whether it contains methods or not), this interface is called a general interface (General interface), the following examples are general interfaces

General interfaces cannot be used to define variables (restricted general interfaces can only be used in generics, and will not affect interface definitions before go1.18)

package main

type Int interface {
int | int8 | int16 | int32 | int64
}

func main() {
var i Int
}
//interface contains type constraints


General interfaces can only be used to define type constraints

some interesting designs

Why did you choose square brackets [] instead of angle brackets <> which are common in other languages.

It is to be consistent with the "built-in generics" of map and slice, so that it will be more coordinated to use. The golang official also answered why they chose [] instead of <>, because the angle brackets will cause ambiguity:

When parsing code within a function, such as v := F, at the point of seeing the < it's ambiguous whether we are seeing a type instantiation or an expression using the < operator. Resolving that requires effectively unbounded lookahead. In general we strive to keep the Go parser simple.
When parsing code inside a function block, something like v := F, when the compiler sees the < symbol, it doesn't know if this is an instantiation of a generic type, or if it's an instance that uses less than number expression. Solving this problem requires an efficient unbounded lookahead. But we now prefer to keep Go's parsing simple enough.

Summarize
Above we have introduced the basic concepts of generics and why generics are needed. Before go1.18, everyone had their own "generics" implementation methods. In the next article, we will analyze the implementation principles of golang generics. Go's support for generics is still very cautious, and the current functions are not very rich.
Going back to the first sentence, generics introduce abstraction, and useless abstraction brings complexity, so you should be very careful in the use of generics.
quote

go.dev/ref/spec



go.googlesource.com/proposal/+/…



go.dev/doc/go1.17_…



go.googlesource.com/proposal/+/…



golang3.eddycjy.com/posts/gener…



segmentfault.com/a/119000004…

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00