perso/syncthing-arm
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

vendor: Mega update all dependencies

Browse Source 
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4080
This commit is contained in:
Jakob Borg
2017-04-05 14:34:41 +00:00
parent 49c1527724
commit a1bcc15458
1354 changed files with 55066 additions and 797850 deletions
Download Patch File Download Diff File Expand all files Collapse all files

145
vendor/github.com/klauspost/cpuid/README.md generated vendored
View File

@@ -1,145 +0,0 @@
# cpuid
Package cpuid provides information about the CPU running the current program.
CPU features are detected on startup, and kept for fast access through the life of the application.
Currently x86 / x64 (AMD64) is supported, and no external C (cgo) code is used, which should make the library very easy to use.
You can access the CPU information by accessing the shared CPU variable of the cpuid library.
Package home: https://github.com/klauspost/cpuid
[![GoDoc][1]][2] [![Build Status][3]][4]
[1]: https://godoc.org/github.com/klauspost/cpuid?status.svg
[2]: https://godoc.org/github.com/klauspost/cpuid
[3]: https://travis-ci.org/klauspost/cpuid.svg
[4]: https://travis-ci.org/klauspost/cpuid
# features
## CPU Instructions
* **CMOV** (i686 CMOV)
* **NX** (NX (No-Execute) bit)
* **AMD3DNOW** (AMD 3DNOW)
* **AMD3DNOWEXT** (AMD 3DNowExt)
* **MMX** (standard MMX)
* **MMXEXT** (SSE integer functions or AMD MMX ext)
* **SSE** (SSE functions)
* **SSE2** (P4 SSE functions)
* **SSE3** (Prescott SSE3 functions)
* **SSSE3** (Conroe SSSE3 functions)
* **SSE4** (Penryn SSE4.1 functions)
* **SSE4A** (AMD Barcelona microarchitecture SSE4a instructions)
* **SSE42** (Nehalem SSE4.2 functions)
* **AVX** (AVX functions)
* **AVX2** (AVX2 functions)
* **FMA3** (Intel FMA 3)
* **FMA4** (Bulldozer FMA4 functions)
* **XOP** (Bulldozer XOP functions)
* **F16C** (Half-precision floating-point conversion)
* **BMI1** (Bit Manipulation Instruction Set 1)
* **BMI2** (Bit Manipulation Instruction Set 2)
* **TBM** (AMD Trailing Bit Manipulation)
* **LZCNT** (LZCNT instruction)
* **POPCNT** (POPCNT instruction)
* **AESNI** (Advanced Encryption Standard New Instructions)
* **CLMUL** (Carry-less Multiplication)
* **HTT** (Hyperthreading (enabled))
* **HLE** (Hardware Lock Elision)
* **RTM** (Restricted Transactional Memory)
* **RDRAND** (RDRAND instruction is available)
* **RDSEED** (RDSEED instruction is available)
* **ADX** (Intel ADX (Multi-Precision Add-Carry Instruction Extensions))
* **SHA** (Intel SHA Extensions)
* **AVX512F** (AVX-512 Foundation)
* **AVX512DQ** (AVX-512 Doubleword and Quadword Instructions)
* **AVX512IFMA** (AVX-512 Integer Fused Multiply-Add Instructions)
* **AVX512PF** (AVX-512 Prefetch Instructions)
* **AVX512ER** (AVX-512 Exponential and Reciprocal Instructions)
* **AVX512CD** (AVX-512 Conflict Detection Instructions)
* **AVX512BW** (AVX-512 Byte and Word Instructions)
* **AVX512VL** (AVX-512 Vector Length Extensions)
* **AVX512VBMI** (AVX-512 Vector Bit Manipulation Instructions)
* **MPX** (Intel MPX (Memory Protection Extensions))
* **ERMS** (Enhanced REP MOVSB/STOSB)
* **RDTSCP** (RDTSCP Instruction)
* **CX16** (CMPXCHG16B Instruction)
* **SGX** (Software Guard Extensions, with activation details)
## Performance
* **RDTSCP()** Returns current cycle count. Can be used for benchmarking.
* **SSE2SLOW** (SSE2 is supported, but usually not faster)
* **SSE3SLOW** (SSE3 is supported, but usually not faster)
* **ATOM** (Atom processor, some SSSE3 instructions are slower)
* **Cache line** (Probable size of a cache line).
* **L1, L2, L3 Cache size** on newer Intel/AMD CPUs.
## Cpu Vendor/VM
* **Intel**
* **AMD**
* **VIA**
* **Transmeta**
* **NSC**
* **KVM** (Kernel-based Virtual Machine)
* **MSVM** (Microsoft Hyper-V or Windows Virtual PC)
* **VMware**
* **XenHVM**
# installing
```go get github.com/klauspost/cpuid```
# example
```Go
package main
import (
"fmt"
"github.com/klauspost/cpuid"
)
func main() {
// Print basic CPU information:
fmt.Println("Name:", cpuid.CPU.BrandName)
fmt.Println("PhysicalCores:", cpuid.CPU.PhysicalCores)
fmt.Println("ThreadsPerCore:", cpuid.CPU.ThreadsPerCore)
fmt.Println("LogicalCores:", cpuid.CPU.LogicalCores)
fmt.Println("Family", cpuid.CPU.Family, "Model:", cpuid.CPU.Model)
fmt.Println("Features:", cpuid.CPU.Features)
fmt.Println("Cacheline bytes:", cpuid.CPU.CacheLine)
fmt.Println("L1 Data Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L1 Instruction Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L2 Cache:", cpuid.CPU.Cache.L2, "bytes")
fmt.Println("L3 Cache:", cpuid.CPU.Cache.L3, "bytes")
// Test if we have a specific feature:
if cpuid.CPU.SSE() {
fmt.Println("We have Streaming SIMD Extensions")
}
}
```
Sample output:
```
>go run main.go
Name: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz
PhysicalCores: 2
ThreadsPerCore: 2
LogicalCores: 4
Family 6 Model: 42
Features: CMOV,MMX,MMXEXT,SSE,SSE2,SSE3,SSSE3,SSE4.1,SSE4.2,AVX,AESNI,CLMUL
Cacheline bytes: 64
We have Streaming SIMD Extensions
```
# private package
In the "private" folder you can find an autogenerated version of the library you can include in your own packages.
For this purpose all exports are removed, and functions and constants are lowercased.
This is not a recommended way of using the library, but provided for convenience, if it is difficult for you to use external packages.
# license
This code is published under an MIT license. See LICENSE file for more information.

727
vendor/github.com/klauspost/cpuid/cpuid_test.go generated vendored
View File

@@ -1,727 +0,0 @@
// Copyright (c) 2015 Klaus Post, released under MIT License. See LICENSE file.
package cpuid
import (
"fmt"
"testing"
)
// There is no real way to test a CPU identifier, since results will
// obviously differ on each machine.
func TestCPUID(t *testing.T) {
n := maxFunctionID()
t.Logf("Max Function:0x%x\n", n)
n = maxExtendedFunction()
t.Logf("Max Extended Function:0x%x\n", n)
t.Log("Name:", CPU.BrandName)
t.Log("PhysicalCores:", CPU.PhysicalCores)
t.Log("ThreadsPerCore:", CPU.ThreadsPerCore)
t.Log("LogicalCores:", CPU.LogicalCores)
t.Log("Family", CPU.Family, "Model:", CPU.Model)
t.Log("Features:", CPU.Features)
t.Log("Cacheline bytes:", CPU.CacheLine)
t.Log("L1 Instruction Cache:", CPU.Cache.L1I, "bytes")
t.Log("L1 Data Cache:", CPU.Cache.L1D, "bytes")
t.Log("L2 Cache:", CPU.Cache.L2, "bytes")
t.Log("L3 Cache:", CPU.Cache.L3, "bytes")
if CPU.SSE2() {
t.Log("We have SSE2")
}
}
func TestDumpCPUID(t *testing.T) {
n := int(maxFunctionID())
for i := 0; i <= n; i++ {
a, b, c, d := cpuidex(uint32(i), 0)
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a, b, c, d)
ex := uint32(1)
for {
a2, b2, c2, d2 := cpuidex(uint32(i), ex)
if a2 == a && b2 == b && d2 == d || ex > 50 || a2 == 0 {
break
}
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a2, b2, c2, d2)
a, b, c, d = a2, b2, c2, d2
ex++
}
}
n2 := maxExtendedFunction()
for i := uint32(0x80000000); i <= n2; i++ {
a, b, c, d := cpuid(i)
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a, b, c, d)
}
}
func Example() {
// Print basic CPU information:
fmt.Println("Name:", CPU.BrandName)
fmt.Println("PhysicalCores:", CPU.PhysicalCores)
fmt.Println("ThreadsPerCore:", CPU.ThreadsPerCore)
fmt.Println("LogicalCores:", CPU.LogicalCores)
fmt.Println("Family", CPU.Family, "Model:", CPU.Model)
fmt.Println("Features:", CPU.Features)
fmt.Println("Cacheline bytes:", CPU.CacheLine)
// Test if we have a specific feature:
if CPU.SSE() {
fmt.Println("We have Streaming SIMD Extensions")
}
}
func TestBrandNameZero(t *testing.T) {
if len(CPU.BrandName) > 0 {
// Cut out last byte
last := []byte(CPU.BrandName[len(CPU.BrandName)-1:])
if last[0] == 0 {
t.Fatal("last byte was zero")
} else if last[0] == 32 {
t.Fatal("whitespace wasn't trimmed")
}
}
}
// Generated here: http://play.golang.org/p/mko-0tFt0Q
// TestCmov tests Cmov() function
func TestCmov(t *testing.T) {
got := CPU.Cmov()
expected := CPU.Features&CMOV == CMOV
if got != expected {
t.Fatalf("Cmov: expected %v, got %v", expected, got)
}
t.Log("CMOV Support:", got)
}
// TestAmd3dnow tests Amd3dnow() function
func TestAmd3dnow(t *testing.T) {
got := CPU.Amd3dnow()
expected := CPU.Features&AMD3DNOW == AMD3DNOW
if got != expected {
t.Fatalf("Amd3dnow: expected %v, got %v", expected, got)
}
t.Log("AMD3DNOW Support:", got)
}
// TestAmd3dnowExt tests Amd3dnowExt() function
func TestAmd3dnowExt(t *testing.T) {
got := CPU.Amd3dnowExt()
expected := CPU.Features&AMD3DNOWEXT == AMD3DNOWEXT
if got != expected {
t.Fatalf("Amd3dnowExt: expected %v, got %v", expected, got)
}
t.Log("AMD3DNOWEXT Support:", got)
}
// TestMMX tests MMX() function
func TestMMX(t *testing.T) {
got := CPU.MMX()
expected := CPU.Features&MMX == MMX
if got != expected {
t.Fatalf("MMX: expected %v, got %v", expected, got)
}
t.Log("MMX Support:", got)
}
// TestMMXext tests MMXext() function
func TestMMXext(t *testing.T) {
got := CPU.MMXExt()
expected := CPU.Features&MMXEXT == MMXEXT
if got != expected {
t.Fatalf("MMXExt: expected %v, got %v", expected, got)
}
t.Log("MMXEXT Support:", got)
}
// TestSSE tests SSE() function
func TestSSE(t *testing.T) {
got := CPU.SSE()
expected := CPU.Features&SSE == SSE
if got != expected {
t.Fatalf("SSE: expected %v, got %v", expected, got)
}
t.Log("SSE Support:", got)
}
// TestSSE2 tests SSE2() function
func TestSSE2(t *testing.T) {
got := CPU.SSE2()
expected := CPU.Features&SSE2 == SSE2
if got != expected {
t.Fatalf("SSE2: expected %v, got %v", expected, got)
}
t.Log("SSE2 Support:", got)
}
// TestSSE3 tests SSE3() function
func TestSSE3(t *testing.T) {
got := CPU.SSE3()
expected := CPU.Features&SSE3 == SSE3
if got != expected {
t.Fatalf("SSE3: expected %v, got %v", expected, got)
}
t.Log("SSE3 Support:", got)
}
// TestSSSE3 tests SSSE3() function
func TestSSSE3(t *testing.T) {
got := CPU.SSSE3()
expected := CPU.Features&SSSE3 == SSSE3
if got != expected {
t.Fatalf("SSSE3: expected %v, got %v", expected, got)
}
t.Log("SSSE3 Support:", got)
}
// TestSSE4 tests SSE4() function
func TestSSE4(t *testing.T) {
got := CPU.SSE4()
expected := CPU.Features&SSE4 == SSE4
if got != expected {
t.Fatalf("SSE4: expected %v, got %v", expected, got)
}
t.Log("SSE4 Support:", got)
}
// TestSSE42 tests SSE42() function
func TestSSE42(t *testing.T) {
got := CPU.SSE42()
expected := CPU.Features&SSE42 == SSE42
if got != expected {
t.Fatalf("SSE42: expected %v, got %v", expected, got)
}
t.Log("SSE42 Support:", got)
}
// TestAVX tests AVX() function
func TestAVX(t *testing.T) {
got := CPU.AVX()
expected := CPU.Features&AVX == AVX
if got != expected {
t.Fatalf("AVX: expected %v, got %v", expected, got)
}
t.Log("AVX Support:", got)
}
// TestAVX2 tests AVX2() function
func TestAVX2(t *testing.T) {
got := CPU.AVX2()
expected := CPU.Features&AVX2 == AVX2
if got != expected {
t.Fatalf("AVX2: expected %v, got %v", expected, got)
}
t.Log("AVX2 Support:", got)
}
// TestFMA3 tests FMA3() function
func TestFMA3(t *testing.T) {
got := CPU.FMA3()
expected := CPU.Features&FMA3 == FMA3
if got != expected {
t.Fatalf("FMA3: expected %v, got %v", expected, got)
}
t.Log("FMA3 Support:", got)
}
// TestFMA4 tests FMA4() function
func TestFMA4(t *testing.T) {
got := CPU.FMA4()
expected := CPU.Features&FMA4 == FMA4
if got != expected {
t.Fatalf("FMA4: expected %v, got %v", expected, got)
}
t.Log("FMA4 Support:", got)
}
// TestXOP tests XOP() function
func TestXOP(t *testing.T) {
got := CPU.XOP()
expected := CPU.Features&XOP == XOP
if got != expected {
t.Fatalf("XOP: expected %v, got %v", expected, got)
}
t.Log("XOP Support:", got)
}
// TestF16C tests F16C() function
func TestF16C(t *testing.T) {
got := CPU.F16C()
expected := CPU.Features&F16C == F16C
if got != expected {
t.Fatalf("F16C: expected %v, got %v", expected, got)
}
t.Log("F16C Support:", got)
}
// TestCX16 tests CX16() function
func TestCX16(t *testing.T) {
got := CPU.CX16()
expected := CPU.Features&CX16 == CX16
if got != expected {
t.Fatalf("CX16: expected %v, got %v", expected, got)
}
t.Log("CX16 Support:", got)
}
// TestSGX tests SGX() function
func TestSGX(t *testing.T) {
got := CPU.SGX.Available
expected := CPU.Features&SGX == SGX
if got != expected {
t.Fatalf("SGX: expected %v, got %v", expected, got)
}
t.Log("SGX Support:", got)
}
// TestBMI1 tests BMI1() function
func TestBMI1(t *testing.T) {
got := CPU.BMI1()
expected := CPU.Features&BMI1 == BMI1
if got != expected {
t.Fatalf("BMI1: expected %v, got %v", expected, got)
}
t.Log("BMI1 Support:", got)
}
// TestBMI2 tests BMI2() function
func TestBMI2(t *testing.T) {
got := CPU.BMI2()
expected := CPU.Features&BMI2 == BMI2
if got != expected {
t.Fatalf("BMI2: expected %v, got %v", expected, got)
}
t.Log("BMI2 Support:", got)
}
// TestTBM tests TBM() function
func TestTBM(t *testing.T) {
got := CPU.TBM()
expected := CPU.Features&TBM == TBM
if got != expected {
t.Fatalf("TBM: expected %v, got %v", expected, got)
}
t.Log("TBM Support:", got)
}
// TestLzcnt tests Lzcnt() function
func TestLzcnt(t *testing.T) {
got := CPU.Lzcnt()
expected := CPU.Features&LZCNT == LZCNT
if got != expected {
t.Fatalf("Lzcnt: expected %v, got %v", expected, got)
}
t.Log("LZCNT Support:", got)
}
// TestLzcnt tests Lzcnt() function
func TestPopcnt(t *testing.T) {
got := CPU.Popcnt()
expected := CPU.Features&POPCNT == POPCNT
if got != expected {
t.Fatalf("Popcnt: expected %v, got %v", expected, got)
}
t.Log("POPCNT Support:", got)
}
// TestAesNi tests AesNi() function
func TestAesNi(t *testing.T) {
got := CPU.AesNi()
expected := CPU.Features&AESNI == AESNI
if got != expected {
t.Fatalf("AesNi: expected %v, got %v", expected, got)
}
t.Log("AESNI Support:", got)
}
// TestHTT tests HTT() function
func TestHTT(t *testing.T) {
got := CPU.HTT()
expected := CPU.Features&HTT == HTT
if got != expected {
t.Fatalf("HTT: expected %v, got %v", expected, got)
}
t.Log("HTT Support:", got)
}
// TestClmul tests Clmul() function
func TestClmul(t *testing.T) {
got := CPU.Clmul()
expected := CPU.Features&CLMUL == CLMUL
if got != expected {
t.Fatalf("Clmul: expected %v, got %v", expected, got)
}
t.Log("CLMUL Support:", got)
}
// TestSSE2Slow tests SSE2Slow() function
func TestSSE2Slow(t *testing.T) {
got := CPU.SSE2Slow()
expected := CPU.Features&SSE2SLOW == SSE2SLOW
if got != expected {
t.Fatalf("SSE2Slow: expected %v, got %v", expected, got)
}
t.Log("SSE2SLOW Support:", got)
}
// TestSSE3Slow tests SSE3slow() function
func TestSSE3Slow(t *testing.T) {
got := CPU.SSE3Slow()
expected := CPU.Features&SSE3SLOW == SSE3SLOW
if got != expected {
t.Fatalf("SSE3slow: expected %v, got %v", expected, got)
}
t.Log("SSE3SLOW Support:", got)
}
// TestAtom tests Atom() function
func TestAtom(t *testing.T) {
got := CPU.Atom()
expected := CPU.Features&ATOM == ATOM
if got != expected {
t.Fatalf("Atom: expected %v, got %v", expected, got)
}
t.Log("ATOM Support:", got)
}
// TestNX tests NX() function (NX (No-Execute) bit)
func TestNX(t *testing.T) {
got := CPU.NX()
expected := CPU.Features&NX == NX
if got != expected {
t.Fatalf("NX: expected %v, got %v", expected, got)
}
t.Log("NX Support:", got)
}
// TestSSE4A tests SSE4A() function (AMD Barcelona microarchitecture SSE4a instructions)
func TestSSE4A(t *testing.T) {
got := CPU.SSE4A()
expected := CPU.Features&SSE4A == SSE4A
if got != expected {
t.Fatalf("SSE4A: expected %v, got %v", expected, got)
}
t.Log("SSE4A Support:", got)
}
// TestHLE tests HLE() function (Hardware Lock Elision)
func TestHLE(t *testing.T) {
got := CPU.HLE()
expected := CPU.Features&HLE == HLE
if got != expected {
t.Fatalf("HLE: expected %v, got %v", expected, got)
}
t.Log("HLE Support:", got)
}
// TestRTM tests RTM() function (Restricted Transactional Memory)
func TestRTM(t *testing.T) {
got := CPU.RTM()
expected := CPU.Features&RTM == RTM
if got != expected {
t.Fatalf("RTM: expected %v, got %v", expected, got)
}
t.Log("RTM Support:", got)
}
// TestRdrand tests RDRAND() function (RDRAND instruction is available)
func TestRdrand(t *testing.T) {
got := CPU.Rdrand()
expected := CPU.Features&RDRAND == RDRAND
if got != expected {
t.Fatalf("Rdrand: expected %v, got %v", expected, got)
}
t.Log("Rdrand Support:", got)
}
// TestRdseed tests RDSEED() function (RDSEED instruction is available)
func TestRdseed(t *testing.T) {
got := CPU.Rdseed()
expected := CPU.Features&RDSEED == RDSEED
if got != expected {
t.Fatalf("Rdseed: expected %v, got %v", expected, got)
}
t.Log("Rdseed Support:", got)
}
// TestADX tests ADX() function (Intel ADX (Multi-Precision Add-Carry Instruction Extensions))
func TestADX(t *testing.T) {
got := CPU.ADX()
expected := CPU.Features&ADX == ADX
if got != expected {
t.Fatalf("ADX: expected %v, got %v", expected, got)
}
t.Log("ADX Support:", got)
}
// TestSHA tests SHA() function (Intel SHA Extensions)
func TestSHA(t *testing.T) {
got := CPU.SHA()
expected := CPU.Features&SHA == SHA
if got != expected {
t.Fatalf("SHA: expected %v, got %v", expected, got)
}
t.Log("SHA Support:", got)
}
// TestAVX512F tests AVX512F() function (AVX-512 Foundation)
func TestAVX512F(t *testing.T) {
got := CPU.AVX512F()
expected := CPU.Features&AVX512F == AVX512F
if got != expected {
t.Fatalf("AVX512F: expected %v, got %v", expected, got)
}
t.Log("AVX512F Support:", got)
}
// TestAVX512DQ tests AVX512DQ() function (AVX-512 Doubleword and Quadword Instructions)
func TestAVX512DQ(t *testing.T) {
got := CPU.AVX512DQ()
expected := CPU.Features&AVX512DQ == AVX512DQ
if got != expected {
t.Fatalf("AVX512DQ: expected %v, got %v", expected, got)
}
t.Log("AVX512DQ Support:", got)
}
// TestAVX512IFMA tests AVX512IFMA() function (AVX-512 Integer Fused Multiply-Add Instructions)
func TestAVX512IFMA(t *testing.T) {
got := CPU.AVX512IFMA()
expected := CPU.Features&AVX512IFMA == AVX512IFMA
if got != expected {
t.Fatalf("AVX512IFMA: expected %v, got %v", expected, got)
}
t.Log("AVX512IFMA Support:", got)
}
// TestAVX512PF tests AVX512PF() function (AVX-512 Prefetch Instructions)
func TestAVX512PF(t *testing.T) {
got := CPU.AVX512PF()
expected := CPU.Features&AVX512PF == AVX512PF
if got != expected {
t.Fatalf("AVX512PF: expected %v, got %v", expected, got)
}
t.Log("AVX512PF Support:", got)
}
// TestAVX512ER tests AVX512ER() function (AVX-512 Exponential and Reciprocal Instructions)
func TestAVX512ER(t *testing.T) {
got := CPU.AVX512ER()
expected := CPU.Features&AVX512ER == AVX512ER
if got != expected {
t.Fatalf("AVX512ER: expected %v, got %v", expected, got)
}
t.Log("AVX512ER Support:", got)
}
// TestAVX512CD tests AVX512CD() function (AVX-512 Conflict Detection Instructions)
func TestAVX512CD(t *testing.T) {
got := CPU.AVX512CD()
expected := CPU.Features&AVX512CD == AVX512CD
if got != expected {
t.Fatalf("AVX512CD: expected %v, got %v", expected, got)
}
t.Log("AVX512CD Support:", got)
}
// TestAVX512BW tests AVX512BW() function (AVX-512 Byte and Word Instructions)
func TestAVX512BW(t *testing.T) {
got := CPU.AVX512BW()
expected := CPU.Features&AVX512BW == AVX512BW
if got != expected {
t.Fatalf("AVX512BW: expected %v, got %v", expected, got)
}
t.Log("AVX512BW Support:", got)
}
// TestAVX512VL tests AVX512VL() function (AVX-512 Vector Length Extensions)
func TestAVX512VL(t *testing.T) {
got := CPU.AVX512VL()
expected := CPU.Features&AVX512VL == AVX512VL
if got != expected {
t.Fatalf("AVX512VL: expected %v, got %v", expected, got)
}
t.Log("AVX512VL Support:", got)
}
// TestAVX512VL tests AVX512VBMI() function (AVX-512 Vector Bit Manipulation Instructions)
func TestAVX512VBMI(t *testing.T) {
got := CPU.AVX512VBMI()
expected := CPU.Features&AVX512VBMI == AVX512VBMI
if got != expected {
t.Fatalf("AVX512VBMI: expected %v, got %v", expected, got)
}
t.Log("AVX512VBMI Support:", got)
}
// TestMPX tests MPX() function (Intel MPX (Memory Protection Extensions))
func TestMPX(t *testing.T) {
got := CPU.MPX()
expected := CPU.Features&MPX == MPX
if got != expected {
t.Fatalf("MPX: expected %v, got %v", expected, got)
}
t.Log("MPX Support:", got)
}
// TestERMS tests ERMS() function (Enhanced REP MOVSB/STOSB)
func TestERMS(t *testing.T) {
got := CPU.ERMS()
expected := CPU.Features&ERMS == ERMS
if got != expected {
t.Fatalf("ERMS: expected %v, got %v", expected, got)
}
t.Log("ERMS Support:", got)
}
// TestVendor writes the detected vendor. Will be 0 if unknown
func TestVendor(t *testing.T) {
t.Log("Vendor ID:", CPU.VendorID)
}
// Intel returns true if vendor is recognized as Intel
func TestIntel(t *testing.T) {
got := CPU.Intel()
expected := CPU.VendorID == Intel
if got != expected {
t.Fatalf("TestIntel: expected %v, got %v", expected, got)
}
t.Log("TestIntel:", got)
}
// AMD returns true if vendor is recognized as AMD
func TestAMD(t *testing.T) {
got := CPU.AMD()
expected := CPU.VendorID == AMD
if got != expected {
t.Fatalf("TestAMD: expected %v, got %v", expected, got)
}
t.Log("TestAMD:", got)
}
// Transmeta returns true if vendor is recognized as Transmeta
func TestTransmeta(t *testing.T) {
got := CPU.Transmeta()
expected := CPU.VendorID == Transmeta
if got != expected {
t.Fatalf("TestTransmeta: expected %v, got %v", expected, got)
}
t.Log("TestTransmeta:", got)
}
// NSC returns true if vendor is recognized as National Semiconductor
func TestNSC(t *testing.T) {
got := CPU.NSC()
expected := CPU.VendorID == NSC
if got != expected {
t.Fatalf("TestNSC: expected %v, got %v", expected, got)
}
t.Log("TestNSC:", got)
}
// VIA returns true if vendor is recognized as VIA
func TestVIA(t *testing.T) {
got := CPU.VIA()
expected := CPU.VendorID == VIA
if got != expected {
t.Fatalf("TestVIA: expected %v, got %v", expected, got)
}
t.Log("TestVIA:", got)
}
// Test VM function
func TestVM(t *testing.T) {
t.Log("Vendor ID:", CPU.VM())
}
// Test RTCounter function
func TestRtCounter(t *testing.T) {
a := CPU.RTCounter()
b := CPU.RTCounter()
t.Log("CPU Counter:", a, b, b-a)
}
// Prints the value of Ia32TscAux()
func TestIa32TscAux(t *testing.T) {
ecx := CPU.Ia32TscAux()
t.Logf("Ia32TscAux:0x%x\n", ecx)
if ecx != 0 {
chip := (ecx & 0xFFF000) >> 12
core := ecx & 0xFFF
t.Log("Likely chip, core:", chip, core)
}
}
func TestThreadsPerCoreNZ(t *testing.T) {
if CPU.ThreadsPerCore == 0 {
t.Fatal("threads per core is zero")
}
}
// Prints the value of LogicalCPU()
func TestLogicalCPU(t *testing.T) {
t.Log("Currently executing on cpu:", CPU.LogicalCPU())
}
func TestMaxFunction(t *testing.T) {
expect := maxFunctionID()
if CPU.maxFunc != expect {
t.Fatal("Max function does not match, expected", expect, "but got", CPU.maxFunc)
}
expect = maxExtendedFunction()
if CPU.maxExFunc != expect {
t.Fatal("Max Extended function does not match, expected", expect, "but got", CPU.maxFunc)
}
}
// This example will calculate the chip/core number on Linux
// Linux encodes numa id (<<12) and core id (8bit) into TSC_AUX.
func ExampleCPUInfo_Ia32TscAux(t *testing.T) {
ecx := CPU.Ia32TscAux()
if ecx == 0 {
fmt.Println("Unknown CPU ID")
return
}
chip := (ecx & 0xFFF000) >> 12
core := ecx & 0xFFF
fmt.Println("Chip, Core:", chip, core)
}
/*
func TestPhysical(t *testing.T) {
var test16 = "CPUID 00000000: 0000000d-756e6547-6c65746e-49656e69 \nCPUID 00000001: 000206d7-03200800-1fbee3ff-bfebfbff \nCPUID 00000002: 76035a01-00f0b2ff-00000000-00ca0000 \nCPUID 00000003: 00000000-00000000-00000000-00000000 \nCPUID 00000004: 3c004121-01c0003f-0000003f-00000000 \nCPUID 00000004: 3c004122-01c0003f-0000003f-00000000 \nCPUID 00000004: 3c004143-01c0003f-000001ff-00000000 \nCPUID 00000004: 3c07c163-04c0003f-00003fff-00000006 \nCPUID 00000005: 00000040-00000040-00000003-00021120 \nCPUID 00000006: 00000075-00000002-00000009-00000000 \nCPUID 00000007: 00000000-00000000-00000000-00000000 \nCPUID 00000008: 00000000-00000000-00000000-00000000 \nCPUID 00000009: 00000001-00000000-00000000-00000000 \nCPUID 0000000a: 07300403-00000000-00000000-00000603 \nCPUID 0000000b: 00000000-00000000-00000003-00000003 \nCPUID 0000000b: 00000005-00000010-00000201-00000003 \nCPUID 0000000c: 00000000-00000000-00000000-00000000 \nCPUID 0000000d: 00000007-00000340-00000340-00000000 \nCPUID 0000000d: 00000001-00000000-00000000-00000000 \nCPUID 0000000d: 00000100-00000240-00000000-00000000 \nCPUID 80000000: 80000008-00000000-00000000-00000000 \nCPUID 80000001: 00000000-00000000-00000001-2c100800 \nCPUID 80000002: 20202020-49202020-6c65746e-20295228 \nCPUID 80000003: 6e6f6558-20295228-20555043-322d3545 \nCPUID 80000004: 20303636-20402030-30322e32-007a4847 \nCPUID 80000005: 00000000-00000000-00000000-00000000 \nCPUID 80000006: 00000000-00000000-01006040-00000000 \nCPUID 80000007: 00000000-00000000-00000000-00000100 \nCPUID 80000008: 0000302e-00000000-00000000-00000000"
restore := mockCPU([]byte(test16))
Detect()
t.Log("Name:", CPU.BrandName)
n := maxFunctionID()
t.Logf("Max Function:0x%x\n", n)
n = maxExtendedFunction()
t.Logf("Max Extended Function:0x%x\n", n)
t.Log("PhysicalCores:", CPU.PhysicalCores)
t.Log("ThreadsPerCore:", CPU.ThreadsPerCore)
t.Log("LogicalCores:", CPU.LogicalCores)
t.Log("Family", CPU.Family, "Model:", CPU.Model)
t.Log("Features:", CPU.Features)
t.Log("Cacheline bytes:", CPU.CacheLine)
t.Log("L1 Instruction Cache:", CPU.Cache.L1I, "bytes")
t.Log("L1 Data Cache:", CPU.Cache.L1D, "bytes")
t.Log("L2 Cache:", CPU.Cache.L2, "bytes")
t.Log("L3 Cache:", CPU.Cache.L3, "bytes")
if CPU.LogicalCores > 0 && CPU.PhysicalCores > 0 {
if CPU.LogicalCores != CPU.PhysicalCores*CPU.ThreadsPerCore {
t.Fatalf("Core count mismatch, LogicalCores (%d) != PhysicalCores (%d) * CPU.ThreadsPerCore (%d)",
CPU.LogicalCores, CPU.PhysicalCores, CPU.ThreadsPerCore)
}
}
if CPU.ThreadsPerCore > 1 && !CPU.HTT() {
t.Fatalf("Hyperthreading not detected")
}
if CPU.ThreadsPerCore == 1 && CPU.HTT() {
t.Fatalf("Hyperthreading detected, but only 1 Thread per core")
}
restore()
Detect()
TestCPUID(t)
}
*/

209
vendor/github.com/klauspost/cpuid/mockcpu_test.go generated vendored
View File

@@ -1,209 +0,0 @@
package cpuid
import (
"archive/zip"
"fmt"
"io/ioutil"
"sort"
"strings"
"testing"
)
type fakecpuid map[uint32][][]uint32
type idfuncs struct {
cpuid func(op uint32) (eax, ebx, ecx, edx uint32)
cpuidex func(op, op2 uint32) (eax, ebx, ecx, edx uint32)
xgetbv func(index uint32) (eax, edx uint32)
}
func (f fakecpuid) String() string {
var out = make([]string, 0, len(f))
for key, val := range f {
for _, v := range val {
out = append(out, fmt.Sprintf("CPUID %08x: [%08x, %08x, %08x, %08x]", key, v[0], v[1], v[2], v[3]))
}
}
sorter := sort.StringSlice(out)
sort.Sort(&sorter)
return strings.Join(sorter, "\n")
}
func mockCPU(def []byte) func() {
lines := strings.Split(string(def), "\n")
anyfound := false
fakeID := make(fakecpuid)
for _, line := range lines {
line = strings.Trim(line, "\r\t ")
if !strings.HasPrefix(line, "CPUID") {
continue
}
// Only collect for first cpu
if strings.HasPrefix(line, "CPUID 00000000") {
if anyfound {
break
}
}
if !strings.Contains(line, "-") {
//continue
}
items := strings.Split(line, ":")
if len(items) < 2 {
if len(line) == 51 || len(line) == 50 {
items = []string{line[0:14], line[15:]}
} else {
items = strings.Split(line, "\t")
if len(items) != 2 {
//fmt.Println("not found:", line, "len:", len(line))
continue
}
}
}
items = items[0:2]
vals := strings.Trim(items[1], "\r\n ")
var idV uint32
n, err := fmt.Sscanf(items[0], "CPUID %x", &idV)
if err != nil || n != 1 {
continue
}
existing, ok := fakeID[idV]
if !ok {
existing = make([][]uint32, 0)
}
values := make([]uint32, 4)
n, err = fmt.Sscanf(vals, "%x-%x-%x-%x", &values[0], &values[1], &values[2], &values[3])
if n != 4 || err != nil {
n, err = fmt.Sscanf(vals, "%x %x %x %x", &values[0], &values[1], &values[2], &values[3])
if n != 4 || err != nil {
//fmt.Println("scanned", vals, "got", n, "Err:", err)
continue
}
}
existing = append(existing, values)
fakeID[idV] = existing
anyfound = true
}
restorer := func(f idfuncs) func() {
return func() {
cpuid = f.cpuid
cpuidex = f.cpuidex
xgetbv = f.xgetbv
}
}(idfuncs{cpuid: cpuid, cpuidex: cpuidex, xgetbv: xgetbv})
cpuid = func(op uint32) (eax, ebx, ecx, edx uint32) {
if op == 0x80000000 || op == 0 {
var ok bool
_, ok = fakeID[op]
if !ok {
return 0, 0, 0, 0
}
}
first, ok := fakeID[op]
if !ok {
if op > maxFunctionID() {
panic(fmt.Sprintf("Base not found: %v, request:%#v\n", fakeID, op))
} else {
// we have some entries missing
return 0, 0, 0, 0
}
}
theid := first[0]
return theid[0], theid[1], theid[2], theid[3]
}
cpuidex = func(op, op2 uint32) (eax, ebx, ecx, edx uint32) {
if op == 0x80000000 {
var ok bool
_, ok = fakeID[op]
if !ok {
return 0, 0, 0, 0
}
}
first, ok := fakeID[op]
if !ok {
if op > maxExtendedFunction() {
panic(fmt.Sprintf("Extended not found Info: %v, request:%#v, %#v\n", fakeID, op, op2))
} else {
// we have some entries missing
return 0, 0, 0, 0
}
}
if int(op2) >= len(first) {
//fmt.Printf("Extended not found Info: %v, request:%#v, %#v\n", fakeID, op, op2)
return 0, 0, 0, 0
}
theid := first[op2]
return theid[0], theid[1], theid[2], theid[3]
}
xgetbv = func(index uint32) (eax, edx uint32) {
first, ok := fakeID[1]
if !ok {
panic(fmt.Sprintf("XGETBV not supported %v", fakeID))
}
second := first[0]
// ECX bit 26 must be set
if (second[2] & 1 << 26) == 0 {
panic(fmt.Sprintf("XGETBV not supported %v", fakeID))
}
// We don't have any data to return, unfortunately
return 0, 0
}
return restorer
}
func TestMocks(t *testing.T) {
zr, err := zip.OpenReader("testdata/cpuid_data.zip")
if err != nil {
t.Skip("No testdata:", err)
}
defer zr.Close()
for _, f := range zr.File {
rc, err := f.Open()
if err != nil {
t.Fatal(err)
}
content, err := ioutil.ReadAll(rc)
if err != nil {
t.Fatal(err)
}
rc.Close()
t.Log("Opening", f.FileInfo().Name())
restore := mockCPU(content)
Detect()
t.Log("Name:", CPU.BrandName)
n := maxFunctionID()
t.Logf("Max Function:0x%x\n", n)
n = maxExtendedFunction()
t.Logf("Max Extended Function:0x%x\n", n)
t.Log("PhysicalCores:", CPU.PhysicalCores)
t.Log("ThreadsPerCore:", CPU.ThreadsPerCore)
t.Log("LogicalCores:", CPU.LogicalCores)
t.Log("Family", CPU.Family, "Model:", CPU.Model)
t.Log("Features:", CPU.Features)
t.Log("Cacheline bytes:", CPU.CacheLine)
t.Log("L1 Instruction Cache:", CPU.Cache.L1I, "bytes")
t.Log("L1 Data Cache:", CPU.Cache.L1D, "bytes")
t.Log("L2 Cache:", CPU.Cache.L2, "bytes")
t.Log("L3 Cache:", CPU.Cache.L3, "bytes")
if CPU.LogicalCores > 0 && CPU.PhysicalCores > 0 {
if CPU.LogicalCores != CPU.PhysicalCores*CPU.ThreadsPerCore {
t.Fatalf("Core count mismatch, LogicalCores (%d) != PhysicalCores (%d) * CPU.ThreadsPerCore (%d)",
CPU.LogicalCores, CPU.PhysicalCores, CPU.ThreadsPerCore)
}
}
if CPU.ThreadsPerCore > 1 && !CPU.HTT() {
t.Fatalf("Hyperthreading not detected")
}
if CPU.ThreadsPerCore == 1 && CPU.HTT() {
t.Fatalf("Hyperthreading detected, but only 1 Thread per core")
}
restore()
}
Detect()
}

6
vendor/github.com/klauspost/cpuid/private/README.md generated vendored
View File

@@ -1,6 +0,0 @@
# cpuid private
This is a specially converted of the cpuid package, so it can be included in
a package without exporting anything.
Package home: https://github.com/klauspost/cpuid

719
vendor/github.com/klauspost/cpuid/private/cpuid_test.go generated vendored
View File

@@ -1,719 +0,0 @@
// Generated, DO NOT EDIT,
// but copy it to your own project and rename the package.
// See more at http://github.com/klauspost/cpuid
package cpuid
import (
"fmt"
"testing"
)
// There is no real way to test a CPU identifier, since results will
// obviously differ on each machine.
func TestCPUID(t *testing.T) {
n := maxFunctionID()
t.Logf("Max Function:0x%x\n", n)
n = maxExtendedFunction()
t.Logf("Max Extended Function:0x%x\n", n)
t.Log("Name:", cpu.brandname)
t.Log("PhysicalCores:", cpu.physicalcores)
t.Log("ThreadsPerCore:", cpu.threadspercore)
t.Log("LogicalCores:", cpu.logicalcores)
t.Log("Family", cpu.family, "Model:", cpu.model)
t.Log("Features:", cpu.features)
t.Log("Cacheline bytes:", cpu.cacheline)
t.Log("L1 Instruction Cache:", cpu.cache.l1i, "bytes")
t.Log("L1 Data Cache:", cpu.cache.l1d, "bytes")
t.Log("L2 Cache:", cpu.cache.l2, "bytes")
t.Log("L3 Cache:", cpu.cache.l3, "bytes")
if cpu.sse2() {
t.Log("We have SSE2")
}
}
func TestDumpCPUID(t *testing.T) {
n := int(maxFunctionID())
for i := 0; i <= n; i++ {
a, b, c, d := cpuidex(uint32(i), 0)
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a, b, c, d)
ex := uint32(1)
for {
a2, b2, c2, d2 := cpuidex(uint32(i), ex)
if a2 == a && b2 == b && d2 == d || ex > 50 || a2 == 0 {
break
}
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a2, b2, c2, d2)
a, b, c, d = a2, b2, c2, d2
ex++
}
}
n2 := maxExtendedFunction()
for i := uint32(0x80000000); i <= n2; i++ {
a, b, c, d := cpuid(i)
t.Logf("CPUID %08x: %08x-%08x-%08x-%08x", i, a, b, c, d)
}
}
func example() {
// Print basic CPU information:
fmt.Println("Name:", cpu.brandname)
fmt.Println("PhysicalCores:", cpu.physicalcores)
fmt.Println("ThreadsPerCore:", cpu.threadspercore)
fmt.Println("LogicalCores:", cpu.logicalcores)
fmt.Println("Family", cpu.family, "Model:", cpu.model)
fmt.Println("Features:", cpu.features)
fmt.Println("Cacheline bytes:", cpu.cacheline)
// Test if we have a specific feature:
if cpu.sse() {
fmt.Println("We have Streaming SIMD Extensions")
}
}
func TestBrandNameZero(t *testing.T) {
if len(cpu.brandname) > 0 {
// Cut out last byte
last := []byte(cpu.brandname[len(cpu.brandname)-1:])
if last[0] == 0 {
t.Fatal("last byte was zero")
} else if last[0] == 32 {
t.Fatal("whitespace wasn't trimmed")
}
}
}
// Generated here: http://play.golang.org/p/mko-0tFt0Q
// TestCmov tests Cmov() function
func TestCmov(t *testing.T) {
got := cpu.cmov()
expected := cpu.features&cmov == cmov
if got != expected {
t.Fatalf("Cmov: expected %v, got %v", expected, got)
}
t.Log("CMOV Support:", got)
}
// TestAmd3dnow tests Amd3dnow() function
func TestAmd3dnow(t *testing.T) {
got := cpu.amd3dnow()
expected := cpu.features&amd3dnow == amd3dnow
if got != expected {
t.Fatalf("Amd3dnow: expected %v, got %v", expected, got)
}
t.Log("AMD3DNOW Support:", got)
}
// TestAmd3dnowExt tests Amd3dnowExt() function
func TestAmd3dnowExt(t *testing.T) {
got := cpu.amd3dnowext()
expected := cpu.features&amd3dnowext == amd3dnowext
if got != expected {
t.Fatalf("Amd3dnowExt: expected %v, got %v", expected, got)
}
t.Log("AMD3DNOWEXT Support:", got)
}
// TestMMX tests MMX() function
func TestMMX(t *testing.T) {
got := cpu.mmx()
expected := cpu.features&mmx == mmx
if got != expected {
t.Fatalf("MMX: expected %v, got %v", expected, got)
}
t.Log("MMX Support:", got)
}
// TestMMXext tests MMXext() function
func TestMMXext(t *testing.T) {
got := cpu.mmxext()
expected := cpu.features&mmxext == mmxext
if got != expected {
t.Fatalf("MMXExt: expected %v, got %v", expected, got)
}
t.Log("MMXEXT Support:", got)
}
// TestSSE tests SSE() function
func TestSSE(t *testing.T) {
got := cpu.sse()
expected := cpu.features&sse == sse
if got != expected {
t.Fatalf("SSE: expected %v, got %v", expected, got)
}
t.Log("SSE Support:", got)
}
// TestSSE2 tests SSE2() function
func TestSSE2(t *testing.T) {
got := cpu.sse2()
expected := cpu.features&sse2 == sse2
if got != expected {
t.Fatalf("SSE2: expected %v, got %v", expected, got)
}
t.Log("SSE2 Support:", got)
}
// TestSSE3 tests SSE3() function
func TestSSE3(t *testing.T) {
got := cpu.sse3()
expected := cpu.features&sse3 == sse3
if got != expected {
t.Fatalf("SSE3: expected %v, got %v", expected, got)
}
t.Log("SSE3 Support:", got)
}
// TestSSSE3 tests SSSE3() function
func TestSSSE3(t *testing.T) {
got := cpu.ssse3()
expected := cpu.features&ssse3 == ssse3
if got != expected {
t.Fatalf("SSSE3: expected %v, got %v", expected, got)
}
t.Log("SSSE3 Support:", got)
}
// TestSSE4 tests SSE4() function
func TestSSE4(t *testing.T) {
got := cpu.sse4()
expected := cpu.features&sse4 == sse4
if got != expected {
t.Fatalf("SSE4: expected %v, got %v", expected, got)
}
t.Log("SSE4 Support:", got)
}
// TestSSE42 tests SSE42() function
func TestSSE42(t *testing.T) {
got := cpu.sse42()
expected := cpu.features&sse42 == sse42
if got != expected {
t.Fatalf("SSE42: expected %v, got %v", expected, got)
}
t.Log("SSE42 Support:", got)
}
// TestAVX tests AVX() function
func TestAVX(t *testing.T) {
got := cpu.avx()
expected := cpu.features&avx == avx
if got != expected {
t.Fatalf("AVX: expected %v, got %v", expected, got)
}
t.Log("AVX Support:", got)
}
// TestAVX2 tests AVX2() function
func TestAVX2(t *testing.T) {
got := cpu.avx2()
expected := cpu.features&avx2 == avx2
if got != expected {
t.Fatalf("AVX2: expected %v, got %v", expected, got)
}
t.Log("AVX2 Support:", got)
}
// TestFMA3 tests FMA3() function
func TestFMA3(t *testing.T) {
got := cpu.fma3()
expected := cpu.features&fma3 == fma3
if got != expected {
t.Fatalf("FMA3: expected %v, got %v", expected, got)
}
t.Log("FMA3 Support:", got)
}
// TestFMA4 tests FMA4() function
func TestFMA4(t *testing.T) {
got := cpu.fma4()
expected := cpu.features&fma4 == fma4
if got != expected {
t.Fatalf("FMA4: expected %v, got %v", expected, got)
}
t.Log("FMA4 Support:", got)
}
// TestXOP tests XOP() function
func TestXOP(t *testing.T) {
got := cpu.xop()
expected := cpu.features&xop == xop
if got != expected {
t.Fatalf("XOP: expected %v, got %v", expected, got)
}
t.Log("XOP Support:", got)
}
// TestF16C tests F16C() function
func TestF16C(t *testing.T) {
got := cpu.f16c()
expected := cpu.features&f16c == f16c
if got != expected {
t.Fatalf("F16C: expected %v, got %v", expected, got)
}
t.Log("F16C Support:", got)
}
// TestCX16 tests CX16() function
func TestCX16(t *testing.T) {
got := cpu.cx16()
expected := cpu.features&cx16 == cx16
if got != expected {
t.Fatalf("CX16: expected %v, got %v", expected, got)
}
t.Log("CX16 Support:", got)
}
// TestBMI1 tests BMI1() function
func TestBMI1(t *testing.T) {
got := cpu.bmi1()
expected := cpu.features&bmi1 == bmi1
if got != expected {
t.Fatalf("BMI1: expected %v, got %v", expected, got)
}
t.Log("BMI1 Support:", got)
}
// TestBMI2 tests BMI2() function
func TestBMI2(t *testing.T) {
got := cpu.bmi2()
expected := cpu.features&bmi2 == bmi2
if got != expected {
t.Fatalf("BMI2: expected %v, got %v", expected, got)
}
t.Log("BMI2 Support:", got)
}
// TestTBM tests TBM() function
func TestTBM(t *testing.T) {
got := cpu.tbm()
expected := cpu.features&tbm == tbm
if got != expected {
t.Fatalf("TBM: expected %v, got %v", expected, got)
}
t.Log("TBM Support:", got)
}
// TestLzcnt tests Lzcnt() function
func TestLzcnt(t *testing.T) {
got := cpu.lzcnt()
expected := cpu.features&lzcnt == lzcnt
if got != expected {
t.Fatalf("Lzcnt: expected %v, got %v", expected, got)
}
t.Log("LZCNT Support:", got)
}
// TestLzcnt tests Lzcnt() function
func TestPopcnt(t *testing.T) {
got := cpu.popcnt()
expected := cpu.features&popcnt == popcnt
if got != expected {
t.Fatalf("Popcnt: expected %v, got %v", expected, got)
}
t.Log("POPCNT Support:", got)
}
// TestAesNi tests AesNi() function
func TestAesNi(t *testing.T) {
got := cpu.aesni()
expected := cpu.features&aesni == aesni
if got != expected {
t.Fatalf("AesNi: expected %v, got %v", expected, got)
}
t.Log("AESNI Support:", got)
}
// TestHTT tests HTT() function
func TestHTT(t *testing.T) {
got := cpu.htt()
expected := cpu.features&htt == htt
if got != expected {
t.Fatalf("HTT: expected %v, got %v", expected, got)
}
t.Log("HTT Support:", got)
}
// TestClmul tests Clmul() function
func TestClmul(t *testing.T) {
got := cpu.clmul()
expected := cpu.features&clmul == clmul
if got != expected {
t.Fatalf("Clmul: expected %v, got %v", expected, got)
}
t.Log("CLMUL Support:", got)
}
// TestSSE2Slow tests SSE2Slow() function
func TestSSE2Slow(t *testing.T) {
got := cpu.sse2slow()
expected := cpu.features&sse2slow == sse2slow
if got != expected {
t.Fatalf("SSE2Slow: expected %v, got %v", expected, got)
}
t.Log("SSE2SLOW Support:", got)
}
// TestSSE3Slow tests SSE3slow() function
func TestSSE3Slow(t *testing.T) {
got := cpu.sse3slow()
expected := cpu.features&sse3slow == sse3slow
if got != expected {
t.Fatalf("SSE3slow: expected %v, got %v", expected, got)
}
t.Log("SSE3SLOW Support:", got)
}
// TestAtom tests Atom() function
func TestAtom(t *testing.T) {
got := cpu.atom()
expected := cpu.features&atom == atom
if got != expected {
t.Fatalf("Atom: expected %v, got %v", expected, got)
}
t.Log("ATOM Support:", got)
}
// TestNX tests NX() function (NX (No-Execute) bit)
func TestNX(t *testing.T) {
got := cpu.nx()
expected := cpu.features&nx == nx
if got != expected {
t.Fatalf("NX: expected %v, got %v", expected, got)
}
t.Log("NX Support:", got)
}
// TestSSE4A tests SSE4A() function (AMD Barcelona microarchitecture SSE4a instructions)
func TestSSE4A(t *testing.T) {
got := cpu.sse4a()
expected := cpu.features&sse4a == sse4a
if got != expected {
t.Fatalf("SSE4A: expected %v, got %v", expected, got)
}
t.Log("SSE4A Support:", got)
}
// TestHLE tests HLE() function (Hardware Lock Elision)
func TestHLE(t *testing.T) {
got := cpu.hle()
expected := cpu.features&hle == hle
if got != expected {
t.Fatalf("HLE: expected %v, got %v", expected, got)
}
t.Log("HLE Support:", got)
}
// TestRTM tests RTM() function (Restricted Transactional Memory)
func TestRTM(t *testing.T) {
got := cpu.rtm()
expected := cpu.features&rtm == rtm
if got != expected {
t.Fatalf("RTM: expected %v, got %v", expected, got)
}
t.Log("RTM Support:", got)
}
// TestRdrand tests RDRAND() function (RDRAND instruction is available)
func TestRdrand(t *testing.T) {
got := cpu.rdrand()
expected := cpu.features&rdrand == rdrand
if got != expected {
t.Fatalf("Rdrand: expected %v, got %v", expected, got)
}
t.Log("Rdrand Support:", got)
}
// TestRdseed tests RDSEED() function (RDSEED instruction is available)
func TestRdseed(t *testing.T) {
got := cpu.rdseed()
expected := cpu.features&rdseed == rdseed
if got != expected {
t.Fatalf("Rdseed: expected %v, got %v", expected, got)
}
t.Log("Rdseed Support:", got)
}
// TestADX tests ADX() function (Intel ADX (Multi-Precision Add-Carry Instruction Extensions))
func TestADX(t *testing.T) {
got := cpu.adx()
expected := cpu.features&adx == adx
if got != expected {
t.Fatalf("ADX: expected %v, got %v", expected, got)
}
t.Log("ADX Support:", got)
}
// TestSHA tests SHA() function (Intel SHA Extensions)
func TestSHA(t *testing.T) {
got := cpu.sha()
expected := cpu.features&sha == sha
if got != expected {
t.Fatalf("SHA: expected %v, got %v", expected, got)
}
t.Log("SHA Support:", got)
}
// TestAVX512F tests AVX512F() function (AVX-512 Foundation)
func TestAVX512F(t *testing.T) {
got := cpu.avx512f()
expected := cpu.features&avx512f == avx512f
if got != expected {
t.Fatalf("AVX512F: expected %v, got %v", expected, got)
}
t.Log("AVX512F Support:", got)
}
// TestAVX512DQ tests AVX512DQ() function (AVX-512 Doubleword and Quadword Instructions)
func TestAVX512DQ(t *testing.T) {
got := cpu.avx512dq()
expected := cpu.features&avx512dq == avx512dq
if got != expected {
t.Fatalf("AVX512DQ: expected %v, got %v", expected, got)
}
t.Log("AVX512DQ Support:", got)
}
// TestAVX512IFMA tests AVX512IFMA() function (AVX-512 Integer Fused Multiply-Add Instructions)
func TestAVX512IFMA(t *testing.T) {
got := cpu.avx512ifma()
expected := cpu.features&avx512ifma == avx512ifma
if got != expected {
t.Fatalf("AVX512IFMA: expected %v, got %v", expected, got)
}
t.Log("AVX512IFMA Support:", got)
}
// TestAVX512PF tests AVX512PF() function (AVX-512 Prefetch Instructions)
func TestAVX512PF(t *testing.T) {
got := cpu.avx512pf()
expected := cpu.features&avx512pf == avx512pf
if got != expected {
t.Fatalf("AVX512PF: expected %v, got %v", expected, got)
}
t.Log("AVX512PF Support:", got)
}
// TestAVX512ER tests AVX512ER() function (AVX-512 Exponential and Reciprocal Instructions)
func TestAVX512ER(t *testing.T) {
got := cpu.avx512er()
expected := cpu.features&avx512er == avx512er
if got != expected {
t.Fatalf("AVX512ER: expected %v, got %v", expected, got)
}
t.Log("AVX512ER Support:", got)
}
// TestAVX512CD tests AVX512CD() function (AVX-512 Conflict Detection Instructions)
func TestAVX512CD(t *testing.T) {
got := cpu.avx512cd()
expected := cpu.features&avx512cd == avx512cd
if got != expected {
t.Fatalf("AVX512CD: expected %v, got %v", expected, got)
}
t.Log("AVX512CD Support:", got)
}
// TestAVX512BW tests AVX512BW() function (AVX-512 Byte and Word Instructions)
func TestAVX512BW(t *testing.T) {
got := cpu.avx512bw()
expected := cpu.features&avx512bw == avx512bw
if got != expected {
t.Fatalf("AVX512BW: expected %v, got %v", expected, got)
}
t.Log("AVX512BW Support:", got)
}
// TestAVX512VL tests AVX512VL() function (AVX-512 Vector Length Extensions)
func TestAVX512VL(t *testing.T) {
got := cpu.avx512vl()
expected := cpu.features&avx512vl == avx512vl
if got != expected {
t.Fatalf("AVX512VL: expected %v, got %v", expected, got)
}
t.Log("AVX512VL Support:", got)
}
// TestAVX512VL tests AVX512VBMI() function (AVX-512 Vector Bit Manipulation Instructions)
func TestAVX512VBMI(t *testing.T) {
got := cpu.avx512vbmi()
expected := cpu.features&avx512vbmi == avx512vbmi
if got != expected {
t.Fatalf("AVX512VBMI: expected %v, got %v", expected, got)
}
t.Log("AVX512VBMI Support:", got)
}
// TestMPX tests MPX() function (Intel MPX (Memory Protection Extensions))
func TestMPX(t *testing.T) {
got := cpu.mpx()
expected := cpu.features&mpx == mpx
if got != expected {
t.Fatalf("MPX: expected %v, got %v", expected, got)
}
t.Log("MPX Support:", got)
}
// TestERMS tests ERMS() function (Enhanced REP MOVSB/STOSB)
func TestERMS(t *testing.T) {
got := cpu.erms()
expected := cpu.features&erms == erms
if got != expected {
t.Fatalf("ERMS: expected %v, got %v", expected, got)
}
t.Log("ERMS Support:", got)
}
// TestVendor writes the detected vendor. Will be 0 if unknown
func TestVendor(t *testing.T) {
t.Log("Vendor ID:", cpu.vendorid)
}
// Intel returns true if vendor is recognized as Intel
func TestIntel(t *testing.T) {
got := cpu.intel()
expected := cpu.vendorid == intel
if got != expected {
t.Fatalf("TestIntel: expected %v, got %v", expected, got)
}
t.Log("TestIntel:", got)
}
// AMD returns true if vendor is recognized as AMD
func TestAMD(t *testing.T) {
got := cpu.amd()
expected := cpu.vendorid == amd
if got != expected {
t.Fatalf("TestAMD: expected %v, got %v", expected, got)
}
t.Log("TestAMD:", got)
}
// Transmeta returns true if vendor is recognized as Transmeta
func TestTransmeta(t *testing.T) {
got := cpu.transmeta()
expected := cpu.vendorid == transmeta
if got != expected {
t.Fatalf("TestTransmeta: expected %v, got %v", expected, got)
}
t.Log("TestTransmeta:", got)
}
// NSC returns true if vendor is recognized as National Semiconductor
func TestNSC(t *testing.T) {
got := cpu.nsc()
expected := cpu.vendorid == nsc
if got != expected {
t.Fatalf("TestNSC: expected %v, got %v", expected, got)
}
t.Log("TestNSC:", got)
}
// VIA returns true if vendor is recognized as VIA
func TestVIA(t *testing.T) {
got := cpu.via()
expected := cpu.vendorid == via
if got != expected {
t.Fatalf("TestVIA: expected %v, got %v", expected, got)
}
t.Log("TestVIA:", got)
}
// Test VM function
func TestVM(t *testing.T) {
t.Log("Vendor ID:", cpu.vm())
}
// Test RTCounter function
func TestRtCounter(t *testing.T) {
a := cpu.rtcounter()
b := cpu.rtcounter()
t.Log("CPU Counter:", a, b, b-a)
}
// Prints the value of Ia32TscAux()
func TestIa32TscAux(t *testing.T) {
ecx := cpu.ia32tscaux()
t.Logf("Ia32TscAux:0x%x\n", ecx)
if ecx != 0 {
chip := (ecx & 0xFFF000) >> 12
core := ecx & 0xFFF
t.Log("Likely chip, core:", chip, core)
}
}
func TestThreadsPerCoreNZ(t *testing.T) {
if cpu.threadspercore == 0 {
t.Fatal("threads per core is zero")
}
}
// Prints the value of LogicalCPU()
func TestLogicalCPU(t *testing.T) {
t.Log("Currently executing on cpu:", cpu.logicalcpu())
}
func TestMaxFunction(t *testing.T) {
expect := maxFunctionID()
if cpu.maxFunc != expect {
t.Fatal("Max function does not match, expected", expect, "but got", cpu.maxFunc)
}
expect = maxExtendedFunction()
if cpu.maxExFunc != expect {
t.Fatal("Max Extended function does not match, expected", expect, "but got", cpu.maxFunc)
}
}
// This example will calculate the chip/core number on Linux
// Linux encodes numa id (<<12) and core id (8bit) into TSC_AUX.
func examplecpuinfo_ia32tscaux(t *testing.T) {
ecx := cpu.ia32tscaux()
if ecx == 0 {
fmt.Println("Unknown CPU ID")
return
}
chip := (ecx & 0xFFF000) >> 12
core := ecx & 0xFFF
fmt.Println("Chip, Core:", chip, core)
}
/*
func TestPhysical(t *testing.T) {
var test16 = "CPUID 00000000: 0000000d-756e6547-6c65746e-49656e69 \nCPUID 00000001: 000206d7-03200800-1fbee3ff-bfebfbff \nCPUID 00000002: 76035a01-00f0b2ff-00000000-00ca0000 \nCPUID 00000003: 00000000-00000000-00000000-00000000 \nCPUID 00000004: 3c004121-01c0003f-0000003f-00000000 \nCPUID 00000004: 3c004122-01c0003f-0000003f-00000000 \nCPUID 00000004: 3c004143-01c0003f-000001ff-00000000 \nCPUID 00000004: 3c07c163-04c0003f-00003fff-00000006 \nCPUID 00000005: 00000040-00000040-00000003-00021120 \nCPUID 00000006: 00000075-00000002-00000009-00000000 \nCPUID 00000007: 00000000-00000000-00000000-00000000 \nCPUID 00000008: 00000000-00000000-00000000-00000000 \nCPUID 00000009: 00000001-00000000-00000000-00000000 \nCPUID 0000000a: 07300403-00000000-00000000-00000603 \nCPUID 0000000b: 00000000-00000000-00000003-00000003 \nCPUID 0000000b: 00000005-00000010-00000201-00000003 \nCPUID 0000000c: 00000000-00000000-00000000-00000000 \nCPUID 0000000d: 00000007-00000340-00000340-00000000 \nCPUID 0000000d: 00000001-00000000-00000000-00000000 \nCPUID 0000000d: 00000100-00000240-00000000-00000000 \nCPUID 80000000: 80000008-00000000-00000000-00000000 \nCPUID 80000001: 00000000-00000000-00000001-2c100800 \nCPUID 80000002: 20202020-49202020-6c65746e-20295228 \nCPUID 80000003: 6e6f6558-20295228-20555043-322d3545 \nCPUID 80000004: 20303636-20402030-30322e32-007a4847 \nCPUID 80000005: 00000000-00000000-00000000-00000000 \nCPUID 80000006: 00000000-00000000-01006040-00000000 \nCPUID 80000007: 00000000-00000000-00000000-00000100 \nCPUID 80000008: 0000302e-00000000-00000000-00000000"
restore := mockCPU([]byte(test16))
Detect()
t.Log("Name:", CPU.BrandName)
n := maxFunctionID()
t.Logf("Max Function:0x%x\n", n)
n = maxExtendedFunction()
t.Logf("Max Extended Function:0x%x\n", n)
t.Log("PhysicalCores:", CPU.PhysicalCores)
t.Log("ThreadsPerCore:", CPU.ThreadsPerCore)
t.Log("LogicalCores:", CPU.LogicalCores)
t.Log("Family", CPU.Family, "Model:", CPU.Model)
t.Log("Features:", CPU.Features)
t.Log("Cacheline bytes:", CPU.CacheLine)
t.Log("L1 Instruction Cache:", CPU.Cache.L1I, "bytes")
t.Log("L1 Data Cache:", CPU.Cache.L1D, "bytes")
t.Log("L2 Cache:", CPU.Cache.L2, "bytes")
t.Log("L3 Cache:", CPU.Cache.L3, "bytes")
if CPU.LogicalCores > 0 && CPU.PhysicalCores > 0 {
if CPU.LogicalCores != CPU.PhysicalCores*CPU.ThreadsPerCore {
t.Fatalf("Core count mismatch, LogicalCores (%d) != PhysicalCores (%d) * CPU.ThreadsPerCore (%d)",
CPU.LogicalCores, CPU.PhysicalCores, CPU.ThreadsPerCore)
}
}
if CPU.ThreadsPerCore > 1 && !CPU.HTT() {
t.Fatalf("Hyperthreading not detected")
}
if CPU.ThreadsPerCore == 1 && CPU.HTT() {
t.Fatalf("Hyperthreading detected, but only 1 Thread per core")
}
restore()
Detect()
TestCPUID(t)
}
*/

77
vendor/github.com/klauspost/cpuid/testdata/getall.go generated vendored
View File

@@ -1,77 +0,0 @@
package main
import (
"archive/zip"
_ "bytes"
"fmt"
"golang.org/x/net/html"
"io"
"net/http"
"os"
"strings"
)
// Download all CPUID dumps from http://users.atw.hu/instlatx64/
func main() {
resp, err := http.Get("http://users.atw.hu/instlatx64/?")
if err != nil {
panic(err)
}
node, err := html.Parse(resp.Body)
if err != nil {
panic(err)
}
file, err := os.Create("cpuid_data.zip")
if err != nil {
panic(err)
}
defer file.Close()
gw := zip.NewWriter(file)
var f func(*html.Node)
f = func(n *html.Node) {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key == "href" {
err := ParseURL(a.Val, gw)
if err != nil {
panic(err)
}
break
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
f(c)
}
}
f(node)
err = gw.Close()
if err != nil {
panic(err)
}
}
func ParseURL(s string, gw *zip.Writer) error {
if strings.Contains(s, "CPUID.txt") {
fmt.Println("Adding", "http://users.atw.hu/instlatx64/"+s)
resp, err := http.Get("http://users.atw.hu/instlatx64/" + s)
if err != nil {
fmt.Println("Error getting ", s, ":", err)
}
defer resp.Body.Close()
w, err := gw.Create(s)
if err != nil {
return err
}
_, err = io.Copy(w, resp.Body)
if err != nil {
return err
}
}
return nil
}

28
vendor/github.com/klauspost/crc32/LICENSE generated vendored
View File

@@ -1,28 +0,0 @@
Copyright (c) 2012 The Go Authors. All rights reserved.
Copyright (c) 2015 Klaus Post
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

87
vendor/github.com/klauspost/crc32/README.md generated vendored
View File

@@ -1,87 +0,0 @@
# crc32
CRC32 hash with x64 optimizations
This package is a drop-in replacement for the standard library `hash/crc32` package, that features SSE 4.2 optimizations on x64 platforms, for a 10x speedup.
[![Build Status](https://travis-ci.org/klauspost/crc32.svg?branch=master)](https://travis-ci.org/klauspost/crc32)
# usage
Install using `go get github.com/klauspost/crc32`. This library is based on Go 1.5 code and requires Go 1.3 or newer.
Replace `import "hash/crc32"` with `import "github.com/klauspost/crc32"` and you are good to go.
# changes
* Oct 20, 2016: Changes have been merged to upstream Go. Package updated to match.
* Dec 4, 2015: Uses the "slice-by-8" trick more extensively, which gives a 1.5 to 2.5x speedup if assembler is unavailable.
# performance
For *Go 1.7* performance is equivalent to the standard library. So if you use this package for Go 1.7 you can switch back.
For IEEE tables (the most common), there is approximately a factor 10 speedup with "CLMUL" (Carryless multiplication) instruction:
```
benchmark old ns/op new ns/op delta
BenchmarkCrc32KB 99955 10258 -89.74%
benchmark old MB/s new MB/s speedup
BenchmarkCrc32KB 327.83 3194.20 9.74x
```
For other tables and "CLMUL" capable machines the performance is the same as the standard library.
Here are some detailed benchmarks, comparing to go 1.5 standard library with and without assembler enabled.
```
Std: Standard Go 1.5 library
Crc: Indicates IEEE type CRC.
40B: Size of each slice encoded.
NoAsm: Assembler was disabled (ie. not an AMD64 or SSE 4.2+ capable machine).
Castagnoli: Castagnoli CRC type.
BenchmarkStdCrc40B-4 10000000 158 ns/op 252.88 MB/s
BenchmarkCrc40BNoAsm-4 20000000 105 ns/op 377.38 MB/s (slice8)
BenchmarkCrc40B-4 20000000 105 ns/op 378.77 MB/s (slice8)
BenchmarkStdCrc1KB-4 500000 3604 ns/op 284.10 MB/s
BenchmarkCrc1KBNoAsm-4 1000000 1463 ns/op 699.79 MB/s (slice8)
BenchmarkCrc1KB-4 3000000 396 ns/op 2583.69 MB/s (asm)
BenchmarkStdCrc8KB-4 200000 11417 ns/op 717.48 MB/s (slice8)
BenchmarkCrc8KBNoAsm-4 200000 11317 ns/op 723.85 MB/s (slice8)
BenchmarkCrc8KB-4 500000 2919 ns/op 2805.73 MB/s (asm)
BenchmarkStdCrc32KB-4 30000 45749 ns/op 716.24 MB/s (slice8)
BenchmarkCrc32KBNoAsm-4 30000 45109 ns/op 726.42 MB/s (slice8)
BenchmarkCrc32KB-4 100000 11497 ns/op 2850.09 MB/s (asm)
BenchmarkStdNoAsmCastagnol40B-4 10000000 161 ns/op 246.94 MB/s
BenchmarkStdCastagnoli40B-4 50000000 28.4 ns/op 1410.69 MB/s (asm)
BenchmarkCastagnoli40BNoAsm-4 20000000 100 ns/op 398.01 MB/s (slice8)
BenchmarkCastagnoli40B-4 50000000 28.2 ns/op 1419.54 MB/s (asm)
BenchmarkStdNoAsmCastagnoli1KB-4 500000 3622 ns/op 282.67 MB/s
BenchmarkStdCastagnoli1KB-4 10000000 144 ns/op 7099.78 MB/s (asm)
BenchmarkCastagnoli1KBNoAsm-4 1000000 1475 ns/op 694.14 MB/s (slice8)
BenchmarkCastagnoli1KB-4 10000000 146 ns/op 6993.35 MB/s (asm)
BenchmarkStdNoAsmCastagnoli8KB-4 50000 28781 ns/op 284.63 MB/s
BenchmarkStdCastagnoli8KB-4 1000000 1029 ns/op 7957.89 MB/s (asm)
BenchmarkCastagnoli8KBNoAsm-4 200000 11410 ns/op 717.94 MB/s (slice8)
BenchmarkCastagnoli8KB-4 1000000 1000 ns/op 8188.71 MB/s (asm)
BenchmarkStdNoAsmCastagnoli32KB-4 10000 115426 ns/op 283.89 MB/s
BenchmarkStdCastagnoli32KB-4 300000 4065 ns/op 8059.13 MB/s (asm)
BenchmarkCastagnoli32KBNoAsm-4 30000 45171 ns/op 725.41 MB/s (slice8)
BenchmarkCastagnoli32KB-4 500000 4077 ns/op 8035.89 MB/s (asm)
```
The IEEE assembler optimizations has been submitted and will be part of the Go 1.6 standard library.
However, the improved use of slice-by-8 has not, but will probably be submitted for Go 1.7.
# license
Standard Go license. Changes are Copyright (c) 2015 Klaus Post under same conditions.

207
vendor/github.com/klauspost/crc32/crc32.go generated vendored
View File

@@ -1,207 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package crc32 implements the 32-bit cyclic redundancy check, or CRC-32,
// checksum. See http://en.wikipedia.org/wiki/Cyclic_redundancy_check for
// information.
//
// Polynomials are represented in LSB-first form also known as reversed representation.
//
// See http://en.wikipedia.org/wiki/Mathematics_of_cyclic_redundancy_checks#Reversed_representations_and_reciprocal_polynomials
// for information.
package crc32
import (
"hash"
"sync"
)
// The size of a CRC-32 checksum in bytes.
const Size = 4
// Predefined polynomials.
const (
// IEEE is by far and away the most common CRC-32 polynomial.
// Used by ethernet (IEEE 802.3), v.42, fddi, gzip, zip, png, ...
IEEE = 0xedb88320
// Castagnoli's polynomial, used in iSCSI.
// Has better error detection characteristics than IEEE.
// http://dx.doi.org/10.1109/26.231911
Castagnoli = 0x82f63b78
// Koopman's polynomial.
// Also has better error detection characteristics than IEEE.
// http://dx.doi.org/10.1109/DSN.2002.1028931
Koopman = 0xeb31d82e
)
// Table is a 256-word table representing the polynomial for efficient processing.
type Table [256]uint32
// This file makes use of functions implemented in architecture-specific files.
// The interface that they implement is as follows:
//
// // archAvailableIEEE reports whether an architecture-specific CRC32-IEEE
// // algorithm is available.
// archAvailableIEEE() bool
//
// // archInitIEEE initializes the architecture-specific CRC3-IEEE algorithm.
// // It can only be called if archAvailableIEEE() returns true.
// archInitIEEE()
//
// // archUpdateIEEE updates the given CRC32-IEEE. It can only be called if
// // archInitIEEE() was previously called.
// archUpdateIEEE(crc uint32, p []byte) uint32
//
// // archAvailableCastagnoli reports whether an architecture-specific
// // CRC32-C algorithm is available.
// archAvailableCastagnoli() bool
//
// // archInitCastagnoli initializes the architecture-specific CRC32-C
// // algorithm. It can only be called if archAvailableCastagnoli() returns
// // true.
// archInitCastagnoli()
//
// // archUpdateCastagnoli updates the given CRC32-C. It can only be called
// // if archInitCastagnoli() was previously called.
// archUpdateCastagnoli(crc uint32, p []byte) uint32
// castagnoliTable points to a lazily initialized Table for the Castagnoli
// polynomial. MakeTable will always return this value when asked to make a
// Castagnoli table so we can compare against it to find when the caller is
// using this polynomial.
var castagnoliTable *Table
var castagnoliTable8 *slicing8Table
var castagnoliArchImpl bool
var updateCastagnoli func(crc uint32, p []byte) uint32
var castagnoliOnce sync.Once
func castagnoliInit() {
castagnoliTable = simpleMakeTable(Castagnoli)
castagnoliArchImpl = archAvailableCastagnoli()
if castagnoliArchImpl {
archInitCastagnoli()
updateCastagnoli = archUpdateCastagnoli
} else {
// Initialize the slicing-by-8 table.
castagnoliTable8 = slicingMakeTable(Castagnoli)
updateCastagnoli = func(crc uint32, p []byte) uint32 {
return slicingUpdate(crc, castagnoliTable8, p)
}
}
}
// IEEETable is the table for the IEEE polynomial.
var IEEETable = simpleMakeTable(IEEE)
// ieeeTable8 is the slicing8Table for IEEE
var ieeeTable8 *slicing8Table
var ieeeArchImpl bool
var updateIEEE func(crc uint32, p []byte) uint32
var ieeeOnce sync.Once
func ieeeInit() {
ieeeArchImpl = archAvailableIEEE()
if ieeeArchImpl {
archInitIEEE()
updateIEEE = archUpdateIEEE
} else {
// Initialize the slicing-by-8 table.
ieeeTable8 = slicingMakeTable(IEEE)
updateIEEE = func(crc uint32, p []byte) uint32 {
return slicingUpdate(crc, ieeeTable8, p)
}
}
}
// MakeTable returns a Table constructed from the specified polynomial.
// The contents of this Table must not be modified.
func MakeTable(poly uint32) *Table {
switch poly {
case IEEE:
ieeeOnce.Do(ieeeInit)
return IEEETable
case Castagnoli:
castagnoliOnce.Do(castagnoliInit)
return castagnoliTable
}
return simpleMakeTable(poly)
}
// digest represents the partial evaluation of a checksum.
type digest struct {
crc uint32
tab *Table
}
// New creates a new hash.Hash32 computing the CRC-32 checksum
// using the polynomial represented by the Table.
// Its Sum method will lay the value out in big-endian byte order.
func New(tab *Table) hash.Hash32 {
if tab == IEEETable {
ieeeOnce.Do(ieeeInit)
}
return &digest{0, tab}
}
// NewIEEE creates a new hash.Hash32 computing the CRC-32 checksum
// using the IEEE polynomial.
// Its Sum method will lay the value out in big-endian byte order.
func NewIEEE() hash.Hash32 { return New(IEEETable) }
func (d *digest) Size() int { return Size }
func (d *digest) BlockSize() int { return 1 }
func (d *digest) Reset() { d.crc = 0 }
// Update returns the result of adding the bytes in p to the crc.
func Update(crc uint32, tab *Table, p []byte) uint32 {
switch tab {
case castagnoliTable:
return updateCastagnoli(crc, p)
case IEEETable:
// Unfortunately, because IEEETable is exported, IEEE may be used without a
// call to MakeTable. We have to make sure it gets initialized in that case.
ieeeOnce.Do(ieeeInit)
return updateIEEE(crc, p)
default:
return simpleUpdate(crc, tab, p)
}
}
func (d *digest) Write(p []byte) (n int, err error) {
switch d.tab {
case castagnoliTable:
d.crc = updateCastagnoli(d.crc, p)
case IEEETable:
// We only create digest objects through New() which takes care of
// initialization in this case.
d.crc = updateIEEE(d.crc, p)
default:
d.crc = simpleUpdate(d.crc, d.tab, p)
}
return len(p), nil
}
func (d *digest) Sum32() uint32 { return d.crc }
func (d *digest) Sum(in []byte) []byte {
s := d.Sum32()
return append(in, byte(s>>24), byte(s>>16), byte(s>>8), byte(s))
}
// Checksum returns the CRC-32 checksum of data
// using the polynomial represented by the Table.
func Checksum(data []byte, tab *Table) uint32 { return Update(0, tab, data) }
// ChecksumIEEE returns the CRC-32 checksum of data
// using the IEEE polynomial.
func ChecksumIEEE(data []byte) uint32 {
ieeeOnce.Do(ieeeInit)
return updateIEEE(0, data)
}

230
vendor/github.com/klauspost/crc32/crc32_amd64.go generated vendored
View File

@@ -1,230 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !appengine,!gccgo
// AMD64-specific hardware-assisted CRC32 algorithms. See crc32.go for a
// description of the interface that each architecture-specific file
// implements.
package crc32
import "unsafe"
// This file contains the code to call the SSE 4.2 version of the Castagnoli
// and IEEE CRC.
// haveSSE41/haveSSE42/haveCLMUL are defined in crc_amd64.s and use
// CPUID to test for SSE 4.1, 4.2 and CLMUL support.
func haveSSE41() bool
func haveSSE42() bool
func haveCLMUL() bool
// castagnoliSSE42 is defined in crc32_amd64.s and uses the SSE4.2 CRC32
// instruction.
//go:noescape
func castagnoliSSE42(crc uint32, p []byte) uint32
// castagnoliSSE42Triple is defined in crc32_amd64.s and uses the SSE4.2 CRC32
// instruction.
//go:noescape
func castagnoliSSE42Triple(
crcA, crcB, crcC uint32,
a, b, c []byte,
rounds uint32,
) (retA uint32, retB uint32, retC uint32)
// ieeeCLMUL is defined in crc_amd64.s and uses the PCLMULQDQ
// instruction as well as SSE 4.1.
//go:noescape
func ieeeCLMUL(crc uint32, p []byte) uint32
var sse42 = haveSSE42()
var useFastIEEE = haveCLMUL() && haveSSE41()
const castagnoliK1 = 168
const castagnoliK2 = 1344
type sse42Table [4]Table
var castagnoliSSE42TableK1 *sse42Table
var castagnoliSSE42TableK2 *sse42Table
func archAvailableCastagnoli() bool {
return sse42
}
func archInitCastagnoli() {
if !sse42 {
panic("arch-specific Castagnoli not available")
}
castagnoliSSE42TableK1 = new(sse42Table)
castagnoliSSE42TableK2 = new(sse42Table)
// See description in updateCastagnoli.
// t[0][i] = CRC(i000, O)
// t[1][i] = CRC(0i00, O)
// t[2][i] = CRC(00i0, O)
// t[3][i] = CRC(000i, O)
// where O is a sequence of K zeros.
var tmp [castagnoliK2]byte
for b := 0; b < 4; b++ {
for i := 0; i < 256; i++ {
val := uint32(i) << uint32(b*8)
castagnoliSSE42TableK1[b][i] = castagnoliSSE42(val, tmp[:castagnoliK1])
castagnoliSSE42TableK2[b][i] = castagnoliSSE42(val, tmp[:])
}
}
}
// castagnoliShift computes the CRC32-C of K1 or K2 zeroes (depending on the
// table given) with the given initial crc value. This corresponds to
// CRC(crc, O) in the description in updateCastagnoli.
func castagnoliShift(table *sse42Table, crc uint32) uint32 {
return table[3][crc>>24] ^
table[2][(crc>>16)&0xFF] ^
table[1][(crc>>8)&0xFF] ^
table[0][crc&0xFF]
}
func archUpdateCastagnoli(crc uint32, p []byte) uint32 {
if !sse42 {
panic("not available")
}
// This method is inspired from the algorithm in Intel's white paper:
// "Fast CRC Computation for iSCSI Polynomial Using CRC32 Instruction"
// The same strategy of splitting the buffer in three is used but the
// combining calculation is different; the complete derivation is explained
// below.
//
// -- The basic idea --
//
// The CRC32 instruction (available in SSE4.2) can process 8 bytes at a
// time. In recent Intel architectures the instruction takes 3 cycles;
// however the processor can pipeline up to three instructions if they
// don't depend on each other.
//
// Roughly this means that we can process three buffers in about the same
// time we can process one buffer.
//
// The idea is then to split the buffer in three, CRC the three pieces
// separately and then combine the results.
//
// Combining the results requires precomputed tables, so we must choose a
// fixed buffer length to optimize. The longer the length, the faster; but
// only buffers longer than this length will use the optimization. We choose
// two cutoffs and compute tables for both:
// - one around 512: 168*3=504
// - one around 4KB: 1344*3=4032
//
// -- The nitty gritty --
//
// Let CRC(I, X) be the non-inverted CRC32-C of the sequence X (with
// initial non-inverted CRC I). This function has the following properties:
// (a) CRC(I, AB) = CRC(CRC(I, A), B)
// (b) CRC(I, A xor B) = CRC(I, A) xor CRC(0, B)
//
// Say we want to compute CRC(I, ABC) where A, B, C are three sequences of
// K bytes each, where K is a fixed constant. Let O be the sequence of K zero
// bytes.
//
// CRC(I, ABC) = CRC(I, ABO xor C)
// = CRC(I, ABO) xor CRC(0, C)
// = CRC(CRC(I, AB), O) xor CRC(0, C)
// = CRC(CRC(I, AO xor B), O) xor CRC(0, C)
// = CRC(CRC(I, AO) xor CRC(0, B), O) xor CRC(0, C)
// = CRC(CRC(CRC(I, A), O) xor CRC(0, B), O) xor CRC(0, C)
//
// The castagnoliSSE42Triple function can compute CRC(I, A), CRC(0, B),
// and CRC(0, C) efficiently. We just need to find a way to quickly compute
// CRC(uvwx, O) given a 4-byte initial value uvwx. We can precompute these
// values; since we can't have a 32-bit table, we break it up into four
// 8-bit tables:
//
// CRC(uvwx, O) = CRC(u000, O) xor
// CRC(0v00, O) xor
// CRC(00w0, O) xor
// CRC(000x, O)
//
// We can compute tables corresponding to the four terms for all 8-bit
// values.
crc = ^crc
// If a buffer is long enough to use the optimization, process the first few
// bytes to align the buffer to an 8 byte boundary (if necessary).
if len(p) >= castagnoliK1*3 {
delta := int(uintptr(unsafe.Pointer(&p[0])) & 7)
if delta != 0 {
delta = 8 - delta
crc = castagnoliSSE42(crc, p[:delta])
p = p[delta:]
}
}
// Process 3*K2 at a time.
for len(p) >= castagnoliK2*3 {
// Compute CRC(I, A), CRC(0, B), and CRC(0, C).
crcA, crcB, crcC := castagnoliSSE42Triple(
crc, 0, 0,
p, p[castagnoliK2:], p[castagnoliK2*2:],
castagnoliK2/24)
// CRC(I, AB) = CRC(CRC(I, A), O) xor CRC(0, B)
crcAB := castagnoliShift(castagnoliSSE42TableK2, crcA) ^ crcB
// CRC(I, ABC) = CRC(CRC(I, AB), O) xor CRC(0, C)
crc = castagnoliShift(castagnoliSSE42TableK2, crcAB) ^ crcC
p = p[castagnoliK2*3:]
}
// Process 3*K1 at a time.
for len(p) >= castagnoliK1*3 {
// Compute CRC(I, A), CRC(0, B), and CRC(0, C).
crcA, crcB, crcC := castagnoliSSE42Triple(
crc, 0, 0,
p, p[castagnoliK1:], p[castagnoliK1*2:],
castagnoliK1/24)
// CRC(I, AB) = CRC(CRC(I, A), O) xor CRC(0, B)
crcAB := castagnoliShift(castagnoliSSE42TableK1, crcA) ^ crcB
// CRC(I, ABC) = CRC(CRC(I, AB), O) xor CRC(0, C)
crc = castagnoliShift(castagnoliSSE42TableK1, crcAB) ^ crcC
p = p[castagnoliK1*3:]
}
// Use the simple implementation for what's left.
crc = castagnoliSSE42(crc, p)
return ^crc
}
func archAvailableIEEE() bool {
return useFastIEEE
}
var archIeeeTable8 *slicing8Table
func archInitIEEE() {
if !useFastIEEE {
panic("not available")
}
// We still use slicing-by-8 for small buffers.
archIeeeTable8 = slicingMakeTable(IEEE)
}
func archUpdateIEEE(crc uint32, p []byte) uint32 {
if !useFastIEEE {
panic("not available")
}
if len(p) >= 64 {
left := len(p) & 15
do := len(p) - left
crc = ^ieeeCLMUL(^crc, p[:do])
p = p[do:]
}
if len(p) == 0 {
return crc
}
return slicingUpdate(crc, archIeeeTable8, p)
}

319
vendor/github.com/klauspost/crc32/crc32_amd64.s generated vendored
View File

@@ -1,319 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build gc
#define NOSPLIT 4
#define RODATA 8
// castagnoliSSE42 updates the (non-inverted) crc with the given buffer.
//
// func castagnoliSSE42(crc uint32, p []byte) uint32
TEXT ·castagnoliSSE42(SB), NOSPLIT, $0
MOVL crc+0(FP), AX // CRC value
MOVQ p+8(FP), SI // data pointer
MOVQ p_len+16(FP), CX // len(p)
// If there are fewer than 8 bytes to process, skip alignment.
CMPQ CX, $8
JL less_than_8
MOVQ SI, BX
ANDQ $7, BX
JZ aligned
// Process the first few bytes to 8-byte align the input.
// BX = 8 - BX. We need to process this many bytes to align.
SUBQ $1, BX
XORQ $7, BX
BTQ $0, BX
JNC align_2
CRC32B (SI), AX
DECQ CX
INCQ SI
align_2:
BTQ $1, BX
JNC align_4
// CRC32W (SI), AX
BYTE $0x66; BYTE $0xf2; BYTE $0x0f; BYTE $0x38; BYTE $0xf1; BYTE $0x06
SUBQ $2, CX
ADDQ $2, SI
align_4:
BTQ $2, BX
JNC aligned
// CRC32L (SI), AX
BYTE $0xf2; BYTE $0x0f; BYTE $0x38; BYTE $0xf1; BYTE $0x06
SUBQ $4, CX
ADDQ $4, SI
aligned:
// The input is now 8-byte aligned and we can process 8-byte chunks.
CMPQ CX, $8
JL less_than_8
CRC32Q (SI), AX
ADDQ $8, SI
SUBQ $8, CX
JMP aligned
less_than_8:
// We may have some bytes left over; process 4 bytes, then 2, then 1.
BTQ $2, CX
JNC less_than_4
// CRC32L (SI), AX
BYTE $0xf2; BYTE $0x0f; BYTE $0x38; BYTE $0xf1; BYTE $0x06
ADDQ $4, SI
less_than_4:
BTQ $1, CX
JNC less_than_2
// CRC32W (SI), AX
BYTE $0x66; BYTE $0xf2; BYTE $0x0f; BYTE $0x38; BYTE $0xf1; BYTE $0x06
ADDQ $2, SI
less_than_2:
BTQ $0, CX
JNC done
CRC32B (SI), AX
done:
MOVL AX, ret+32(FP)
RET
// castagnoliSSE42Triple updates three (non-inverted) crcs with (24*rounds)
// bytes from each buffer.
//
// func castagnoliSSE42Triple(
// crc1, crc2, crc3 uint32,
// a, b, c []byte,
// rounds uint32,
// ) (retA uint32, retB uint32, retC uint32)
TEXT ·castagnoliSSE42Triple(SB), NOSPLIT, $0
MOVL crcA+0(FP), AX
MOVL crcB+4(FP), CX
MOVL crcC+8(FP), DX
MOVQ a+16(FP), R8 // data pointer
MOVQ b+40(FP), R9 // data pointer
MOVQ c+64(FP), R10 // data pointer
MOVL rounds+88(FP), R11
loop:
CRC32Q (R8), AX
CRC32Q (R9), CX
CRC32Q (R10), DX
CRC32Q 8(R8), AX
CRC32Q 8(R9), CX
CRC32Q 8(R10), DX
CRC32Q 16(R8), AX
CRC32Q 16(R9), CX
CRC32Q 16(R10), DX
ADDQ $24, R8
ADDQ $24, R9
ADDQ $24, R10
DECQ R11
JNZ loop
MOVL AX, retA+96(FP)
MOVL CX, retB+100(FP)
MOVL DX, retC+104(FP)
RET
// func haveSSE42() bool
TEXT ·haveSSE42(SB), NOSPLIT, $0
XORQ AX, AX
INCL AX
CPUID
SHRQ $20, CX
ANDQ $1, CX
MOVB CX, ret+0(FP)
RET
// func haveCLMUL() bool
TEXT ·haveCLMUL(SB), NOSPLIT, $0
XORQ AX, AX
INCL AX
CPUID
SHRQ $1, CX
ANDQ $1, CX
MOVB CX, ret+0(FP)
RET
// func haveSSE41() bool
TEXT ·haveSSE41(SB), NOSPLIT, $0
XORQ AX, AX
INCL AX
CPUID
SHRQ $19, CX
ANDQ $1, CX
MOVB CX, ret+0(FP)
RET
// CRC32 polynomial data
//
// These constants are lifted from the
// Linux kernel, since they avoid the costly
// PSHUFB 16 byte reversal proposed in the
// original Intel paper.
DATA r2r1kp<>+0(SB)/8, $0x154442bd4
DATA r2r1kp<>+8(SB)/8, $0x1c6e41596
DATA r4r3kp<>+0(SB)/8, $0x1751997d0
DATA r4r3kp<>+8(SB)/8, $0x0ccaa009e
DATA rupolykp<>+0(SB)/8, $0x1db710641
DATA rupolykp<>+8(SB)/8, $0x1f7011641
DATA r5kp<>+0(SB)/8, $0x163cd6124
GLOBL r2r1kp<>(SB), RODATA, $16
GLOBL r4r3kp<>(SB), RODATA, $16
GLOBL rupolykp<>(SB), RODATA, $16
GLOBL r5kp<>(SB), RODATA, $8
// Based on http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf
// len(p) must be at least 64, and must be a multiple of 16.
// func ieeeCLMUL(crc uint32, p []byte) uint32
TEXT ·ieeeCLMUL(SB), NOSPLIT, $0
MOVL crc+0(FP), X0 // Initial CRC value
MOVQ p+8(FP), SI // data pointer
MOVQ p_len+16(FP), CX // len(p)
MOVOU (SI), X1
MOVOU 16(SI), X2
MOVOU 32(SI), X3
MOVOU 48(SI), X4
PXOR X0, X1
ADDQ $64, SI // buf+=64
SUBQ $64, CX // len-=64
CMPQ CX, $64 // Less than 64 bytes left
JB remain64
MOVOA r2r1kp<>+0(SB), X0
loopback64:
MOVOA X1, X5
MOVOA X2, X6
MOVOA X3, X7
MOVOA X4, X8
PCLMULQDQ $0, X0, X1
PCLMULQDQ $0, X0, X2
PCLMULQDQ $0, X0, X3
PCLMULQDQ $0, X0, X4
// Load next early
MOVOU (SI), X11
MOVOU 16(SI), X12
MOVOU 32(SI), X13
MOVOU 48(SI), X14
PCLMULQDQ $0x11, X0, X5
PCLMULQDQ $0x11, X0, X6
PCLMULQDQ $0x11, X0, X7
PCLMULQDQ $0x11, X0, X8
PXOR X5, X1
PXOR X6, X2
PXOR X7, X3
PXOR X8, X4
PXOR X11, X1
PXOR X12, X2
PXOR X13, X3
PXOR X14, X4
ADDQ $0x40, DI
ADDQ $64, SI // buf+=64
SUBQ $64, CX // len-=64
CMPQ CX, $64 // Less than 64 bytes left?
JGE loopback64
// Fold result into a single register (X1)
remain64:
MOVOA r4r3kp<>+0(SB), X0
MOVOA X1, X5
PCLMULQDQ $0, X0, X1
PCLMULQDQ $0x11, X0, X5
PXOR X5, X1
PXOR X2, X1
MOVOA X1, X5
PCLMULQDQ $0, X0, X1
PCLMULQDQ $0x11, X0, X5
PXOR X5, X1
PXOR X3, X1
MOVOA X1, X5
PCLMULQDQ $0, X0, X1
PCLMULQDQ $0x11, X0, X5
PXOR X5, X1
PXOR X4, X1
// If there is less than 16 bytes left we are done
CMPQ CX, $16
JB finish
// Encode 16 bytes
remain16:
MOVOU (SI), X10
MOVOA X1, X5
PCLMULQDQ $0, X0, X1
PCLMULQDQ $0x11, X0, X5
PXOR X5, X1
PXOR X10, X1
SUBQ $16, CX
ADDQ $16, SI
CMPQ CX, $16
JGE remain16
finish:
// Fold final result into 32 bits and return it
PCMPEQB X3, X3
PCLMULQDQ $1, X1, X0
PSRLDQ $8, X1
PXOR X0, X1
MOVOA X1, X2
MOVQ r5kp<>+0(SB), X0
// Creates 32 bit mask. Note that we don't care about upper half.
PSRLQ $32, X3
PSRLDQ $4, X2
PAND X3, X1
PCLMULQDQ $0, X0, X1
PXOR X2, X1
MOVOA rupolykp<>+0(SB), X0
MOVOA X1, X2
PAND X3, X1
PCLMULQDQ $0x10, X0, X1
PAND X3, X1
PCLMULQDQ $0, X0, X1
PXOR X2, X1
// PEXTRD $1, X1, AX (SSE 4.1)
BYTE $0x66; BYTE $0x0f; BYTE $0x3a
BYTE $0x16; BYTE $0xc8; BYTE $0x01
MOVL AX, ret+32(FP)
RET

43
vendor/github.com/klauspost/crc32/crc32_amd64p32.go generated vendored
View File

@@ -1,43 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !appengine,!gccgo
package crc32
// This file contains the code to call the SSE 4.2 version of the Castagnoli
// CRC.
// haveSSE42 is defined in crc32_amd64p32.s and uses CPUID to test for SSE 4.2
// support.
func haveSSE42() bool
// castagnoliSSE42 is defined in crc32_amd64p32.s and uses the SSE4.2 CRC32
// instruction.
//go:noescape
func castagnoliSSE42(crc uint32, p []byte) uint32
var sse42 = haveSSE42()
func archAvailableCastagnoli() bool {
return sse42
}
func archInitCastagnoli() {
if !sse42 {
panic("not available")
}
// No initialization necessary.
}
func archUpdateCastagnoli(crc uint32, p []byte) uint32 {
if !sse42 {
panic("not available")
}
return castagnoliSSE42(crc, p)
}
func archAvailableIEEE() bool { return false }
func archInitIEEE() { panic("not available") }
func archUpdateIEEE(crc uint32, p []byte) uint32 { panic("not available") }

67
vendor/github.com/klauspost/crc32/crc32_amd64p32.s generated vendored
View File

@@ -1,67 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build gc
#define NOSPLIT 4
#define RODATA 8
// func castagnoliSSE42(crc uint32, p []byte) uint32
TEXT ·castagnoliSSE42(SB), NOSPLIT, $0
MOVL crc+0(FP), AX // CRC value
MOVL p+4(FP), SI // data pointer
MOVL p_len+8(FP), CX // len(p)
NOTL AX
// If there's less than 8 bytes to process, we do it byte-by-byte.
CMPQ CX, $8
JL cleanup
// Process individual bytes until the input is 8-byte aligned.
startup:
MOVQ SI, BX
ANDQ $7, BX
JZ aligned
CRC32B (SI), AX
DECQ CX
INCQ SI
JMP startup
aligned:
// The input is now 8-byte aligned and we can process 8-byte chunks.
CMPQ CX, $8
JL cleanup
CRC32Q (SI), AX
ADDQ $8, SI
SUBQ $8, CX
JMP aligned
cleanup:
// We may have some bytes left over that we process one at a time.
CMPQ CX, $0
JE done
CRC32B (SI), AX
INCQ SI
DECQ CX
JMP cleanup
done:
NOTL AX
MOVL AX, ret+16(FP)
RET
// func haveSSE42() bool
TEXT ·haveSSE42(SB), NOSPLIT, $0
XORQ AX, AX
INCL AX
CPUID
SHRQ $20, CX
ANDQ $1, CX
MOVB CX, ret+0(FP)
RET

89
vendor/github.com/klauspost/crc32/crc32_generic.go generated vendored
View File

@@ -1,89 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This file contains CRC32 algorithms that are not specific to any architecture
// and don't use hardware acceleration.
//
// The simple (and slow) CRC32 implementation only uses a 256*4 bytes table.
//
// The slicing-by-8 algorithm is a faster implementation that uses a bigger
// table (8*256*4 bytes).
package crc32
// simpleMakeTable allocates and constructs a Table for the specified
// polynomial. The table is suitable for use with the simple algorithm
// (simpleUpdate).
func simpleMakeTable(poly uint32) *Table {
t := new(Table)
simplePopulateTable(poly, t)
return t
}
// simplePopulateTable constructs a Table for the specified polynomial, suitable
// for use with simpleUpdate.
func simplePopulateTable(poly uint32, t *Table) {
for i := 0; i < 256; i++ {
crc := uint32(i)
for j := 0; j < 8; j++ {
if crc&1 == 1 {
crc = (crc >> 1) ^ poly
} else {
crc >>= 1
}
}
t[i] = crc
}
}
// simpleUpdate uses the simple algorithm to update the CRC, given a table that
// was previously computed using simpleMakeTable.
func simpleUpdate(crc uint32, tab *Table, p []byte) uint32 {
crc = ^crc
for _, v := range p {
crc = tab[byte(crc)^v] ^ (crc >> 8)
}
return ^crc
}
// Use slicing-by-8 when payload >= this value.
const slicing8Cutoff = 16
// slicing8Table is array of 8 Tables, used by the slicing-by-8 algorithm.
type slicing8Table [8]Table
// slicingMakeTable constructs a slicing8Table for the specified polynomial. The
// table is suitable for use with the slicing-by-8 algorithm (slicingUpdate).
func slicingMakeTable(poly uint32) *slicing8Table {
t := new(slicing8Table)
simplePopulateTable(poly, &t[0])
for i := 0; i < 256; i++ {
crc := t[0][i]
for j := 1; j < 8; j++ {
crc = t[0][crc&0xFF] ^ (crc >> 8)
t[j][i] = crc
}
}
return t
}
// slicingUpdate uses the slicing-by-8 algorithm to update the CRC, given a
// table that was previously computed using slicingMakeTable.
func slicingUpdate(crc uint32, tab *slicing8Table, p []byte) uint32 {
if len(p) >= slicing8Cutoff {
crc = ^crc
for len(p) > 8 {
crc ^= uint32(p[0]) | uint32(p[1])<<8 | uint32(p[2])<<16 | uint32(p[3])<<24
crc = tab[0][p[7]] ^ tab[1][p[6]] ^ tab[2][p[5]] ^ tab[3][p[4]] ^
tab[4][crc>>24] ^ tab[5][(crc>>16)&0xFF] ^
tab[6][(crc>>8)&0xFF] ^ tab[7][crc&0xFF]
p = p[8:]
}
crc = ^crc
}
if len(p) == 0 {
return crc
}
return simpleUpdate(crc, &tab[0], p)
}

15
vendor/github.com/klauspost/crc32/crc32_otherarch.go generated vendored
View File

@@ -1,15 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !amd64,!amd64p32,!s390x
package crc32
func archAvailableIEEE() bool { return false }
func archInitIEEE() { panic("not available") }
func archUpdateIEEE(crc uint32, p []byte) uint32 { panic("not available") }
func archAvailableCastagnoli() bool { return false }
func archInitCastagnoli() { panic("not available") }
func archUpdateCastagnoli(crc uint32, p []byte) uint32 { panic("not available") }

91
vendor/github.com/klauspost/crc32/crc32_s390x.go generated vendored
View File

@@ -1,91 +0,0 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build s390x
package crc32
const (
vxMinLen = 64
vxAlignMask = 15 // align to 16 bytes
)
// hasVectorFacility reports whether the machine has the z/Architecture
// vector facility installed and enabled.
func hasVectorFacility() bool
var hasVX = hasVectorFacility()
// vectorizedCastagnoli implements CRC32 using vector instructions.
// It is defined in crc32_s390x.s.
//go:noescape
func vectorizedCastagnoli(crc uint32, p []byte) uint32
// vectorizedIEEE implements CRC32 using vector instructions.
// It is defined in crc32_s390x.s.
//go:noescape
func vectorizedIEEE(crc uint32, p []byte) uint32
func archAvailableCastagnoli() bool {
return hasVX
}
var archCastagnoliTable8 *slicing8Table
func archInitCastagnoli() {
if !hasVX {
panic("not available")
}
// We still use slicing-by-8 for small buffers.
archCastagnoliTable8 = slicingMakeTable(Castagnoli)
}
// archUpdateCastagnoli calculates the checksum of p using
// vectorizedCastagnoli.
func archUpdateCastagnoli(crc uint32, p []byte) uint32 {
if !hasVX {
panic("not available")
}
// Use vectorized function if data length is above threshold.
if len(p) >= vxMinLen {
aligned := len(p) & ^vxAlignMask
crc = vectorizedCastagnoli(crc, p[:aligned])
p = p[aligned:]
}
if len(p) == 0 {
return crc
}
return slicingUpdate(crc, archCastagnoliTable8, p)
}
func archAvailableIEEE() bool {
return hasVX
}
var archIeeeTable8 *slicing8Table
func archInitIEEE() {
if !hasVX {
panic("not available")
}
// We still use slicing-by-8 for small buffers.
archIeeeTable8 = slicingMakeTable(IEEE)
}
// archUpdateIEEE calculates the checksum of p using vectorizedIEEE.
func archUpdateIEEE(crc uint32, p []byte) uint32 {
if !hasVX {
panic("not available")
}
// Use vectorized function if data length is above threshold.
if len(p) >= vxMinLen {
aligned := len(p) & ^vxAlignMask
crc = vectorizedIEEE(crc, p[:aligned])
p = p[aligned:]
}
if len(p) == 0 {
return crc
}
return slicingUpdate(crc, archIeeeTable8, p)
}

249
vendor/github.com/klauspost/crc32/crc32_s390x.s generated vendored
View File

@@ -1,249 +0,0 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build s390x
#include "textflag.h"
// Vector register range containing CRC-32 constants
#define CONST_PERM_LE2BE V9
#define CONST_R2R1 V10
#define CONST_R4R3 V11
#define CONST_R5 V12
#define CONST_RU_POLY V13
#define CONST_CRC_POLY V14
// The CRC-32 constant block contains reduction constants to fold and
// process particular chunks of the input data stream in parallel.
//
// Note that the constant definitions below are extended in order to compute
// intermediate results with a single VECTOR GALOIS FIELD MULTIPLY instruction.
// The rightmost doubleword can be 0 to prevent contribution to the result or
// can be multiplied by 1 to perform an XOR without the need for a separate
// VECTOR EXCLUSIVE OR instruction.
//
// The polynomials used are bit-reflected:
//
// IEEE: P'(x) = 0x0edb88320
// Castagnoli: P'(x) = 0x082f63b78
// IEEE polynomial constants
DATA ·crcleconskp+0(SB)/8, $0x0F0E0D0C0B0A0908 // LE-to-BE mask
DATA ·crcleconskp+8(SB)/8, $0x0706050403020100
DATA ·crcleconskp+16(SB)/8, $0x00000001c6e41596 // R2
DATA ·crcleconskp+24(SB)/8, $0x0000000154442bd4 // R1
DATA ·crcleconskp+32(SB)/8, $0x00000000ccaa009e // R4
DATA ·crcleconskp+40(SB)/8, $0x00000001751997d0 // R3
DATA ·crcleconskp+48(SB)/8, $0x0000000000000000
DATA ·crcleconskp+56(SB)/8, $0x0000000163cd6124 // R5
DATA ·crcleconskp+64(SB)/8, $0x0000000000000000
DATA ·crcleconskp+72(SB)/8, $0x00000001F7011641 // u'
DATA ·crcleconskp+80(SB)/8, $0x0000000000000000
DATA ·crcleconskp+88(SB)/8, $0x00000001DB710641 // P'(x) << 1
GLOBL ·crcleconskp(SB), RODATA, $144
// Castagonli Polynomial constants
DATA ·crccleconskp+0(SB)/8, $0x0F0E0D0C0B0A0908 // LE-to-BE mask
DATA ·crccleconskp+8(SB)/8, $0x0706050403020100
DATA ·crccleconskp+16(SB)/8, $0x000000009e4addf8 // R2
DATA ·crccleconskp+24(SB)/8, $0x00000000740eef02 // R1
DATA ·crccleconskp+32(SB)/8, $0x000000014cd00bd6 // R4
DATA ·crccleconskp+40(SB)/8, $0x00000000f20c0dfe // R3
DATA ·crccleconskp+48(SB)/8, $0x0000000000000000
DATA ·crccleconskp+56(SB)/8, $0x00000000dd45aab8 // R5
DATA ·crccleconskp+64(SB)/8, $0x0000000000000000
DATA ·crccleconskp+72(SB)/8, $0x00000000dea713f1 // u'
DATA ·crccleconskp+80(SB)/8, $0x0000000000000000
DATA ·crccleconskp+88(SB)/8, $0x0000000105ec76f0 // P'(x) << 1
GLOBL ·crccleconskp(SB), RODATA, $144
// func hasVectorFacility() bool
TEXT ·hasVectorFacility(SB), NOSPLIT, $24-1
MOVD $x-24(SP), R1
XC $24, 0(R1), 0(R1) // clear the storage
MOVD $2, R0 // R0 is the number of double words stored -1
WORD $0xB2B01000 // STFLE 0(R1)
XOR R0, R0 // reset the value of R0
MOVBZ z-8(SP), R1
AND $0x40, R1
BEQ novector
vectorinstalled:
// check if the vector instruction has been enabled
VLEIB $0, $0xF, V16
VLGVB $0, V16, R1
CMPBNE R1, $0xF, novector
MOVB $1, ret+0(FP) // have vx
RET
novector:
MOVB $0, ret+0(FP) // no vx
RET
// The CRC-32 function(s) use these calling conventions:
//
// Parameters:
//
// R2: Initial CRC value, typically ~0; and final CRC (return) value.
// R3: Input buffer pointer, performance might be improved if the
// buffer is on a doubleword boundary.
// R4: Length of the buffer, must be 64 bytes or greater.
//
// Register usage:
//
// R5: CRC-32 constant pool base pointer.
// V0: Initial CRC value and intermediate constants and results.
// V1..V4: Data for CRC computation.
// V5..V8: Next data chunks that are fetched from the input buffer.
//
// V9..V14: CRC-32 constants.
// func vectorizedIEEE(crc uint32, p []byte) uint32
TEXT ·vectorizedIEEE(SB), NOSPLIT, $0
MOVWZ crc+0(FP), R2 // R2 stores the CRC value
MOVD p+8(FP), R3 // data pointer
MOVD p_len+16(FP), R4 // len(p)
MOVD $·crcleconskp(SB), R5
BR vectorizedBody<>(SB)
// func vectorizedCastagnoli(crc uint32, p []byte) uint32
TEXT ·vectorizedCastagnoli(SB), NOSPLIT, $0
MOVWZ crc+0(FP), R2 // R2 stores the CRC value
MOVD p+8(FP), R3 // data pointer
MOVD p_len+16(FP), R4 // len(p)
// R5: crc-32 constant pool base pointer, constant is used to reduce crc
MOVD $·crccleconskp(SB), R5
BR vectorizedBody<>(SB)
TEXT vectorizedBody<>(SB), NOSPLIT, $0
XOR $0xffffffff, R2 // NOTW R2
VLM 0(R5), CONST_PERM_LE2BE, CONST_CRC_POLY
// Load the initial CRC value into the rightmost word of V0
VZERO V0
VLVGF $3, R2, V0
// Crash if the input size is less than 64-bytes.
CMP R4, $64
BLT crash
// Load a 64-byte data chunk and XOR with CRC
VLM 0(R3), V1, V4 // 64-bytes into V1..V4
// Reflect the data if the CRC operation is in the bit-reflected domain
VPERM V1, V1, CONST_PERM_LE2BE, V1
VPERM V2, V2, CONST_PERM_LE2BE, V2
VPERM V3, V3, CONST_PERM_LE2BE, V3
VPERM V4, V4, CONST_PERM_LE2BE, V4
VX V0, V1, V1 // V1 ^= CRC
ADD $64, R3 // BUF = BUF + 64
ADD $(-64), R4
// Check remaining buffer size and jump to proper folding method
CMP R4, $64
BLT less_than_64bytes
fold_64bytes_loop:
// Load the next 64-byte data chunk into V5 to V8
VLM 0(R3), V5, V8
VPERM V5, V5, CONST_PERM_LE2BE, V5
VPERM V6, V6, CONST_PERM_LE2BE, V6
VPERM V7, V7, CONST_PERM_LE2BE, V7
VPERM V8, V8, CONST_PERM_LE2BE, V8
// Perform a GF(2) multiplication of the doublewords in V1 with
// the reduction constants in V0. The intermediate result is
// then folded (accumulated) with the next data chunk in V5 and
// stored in V1. Repeat this step for the register contents
// in V2, V3, and V4 respectively.
VGFMAG CONST_R2R1, V1, V5, V1
VGFMAG CONST_R2R1, V2, V6, V2
VGFMAG CONST_R2R1, V3, V7, V3
VGFMAG CONST_R2R1, V4, V8, V4
// Adjust buffer pointer and length for next loop
ADD $64, R3 // BUF = BUF + 64
ADD $(-64), R4 // LEN = LEN - 64
CMP R4, $64
BGE fold_64bytes_loop
less_than_64bytes:
// Fold V1 to V4 into a single 128-bit value in V1
VGFMAG CONST_R4R3, V1, V2, V1
VGFMAG CONST_R4R3, V1, V3, V1
VGFMAG CONST_R4R3, V1, V4, V1
// Check whether to continue with 64-bit folding
CMP R4, $16
BLT final_fold
fold_16bytes_loop:
VL 0(R3), V2 // Load next data chunk
VPERM V2, V2, CONST_PERM_LE2BE, V2
VGFMAG CONST_R4R3, V1, V2, V1 // Fold next data chunk
// Adjust buffer pointer and size for folding next data chunk
ADD $16, R3
ADD $-16, R4
// Process remaining data chunks
CMP R4, $16
BGE fold_16bytes_loop
final_fold:
VLEIB $7, $0x40, V9
VSRLB V9, CONST_R4R3, V0
VLEIG $0, $1, V0
VGFMG V0, V1, V1
VLEIB $7, $0x20, V9 // Shift by words
VSRLB V9, V1, V2 // Store remaining bits in V2
VUPLLF V1, V1 // Split rightmost doubleword
VGFMAG CONST_R5, V1, V2, V1 // V1 = (V1 * R5) XOR V2
// The input values to the Barret reduction are the degree-63 polynomial
// in V1 (R(x)), degree-32 generator polynomial, and the reduction
// constant u. The Barret reduction result is the CRC value of R(x) mod
// P(x).
//
// The Barret reduction algorithm is defined as:
//
// 1. T1(x) = floor( R(x) / x^32 ) GF2MUL u
// 2. T2(x) = floor( T1(x) / x^32 ) GF2MUL P(x)
// 3. C(x) = R(x) XOR T2(x) mod x^32
//
// Note: To compensate the division by x^32, use the vector unpack
// instruction to move the leftmost word into the leftmost doubleword
// of the vector register. The rightmost doubleword is multiplied
// with zero to not contribute to the intermedate results.
// T1(x) = floor( R(x) / x^32 ) GF2MUL u
VUPLLF V1, V2
VGFMG CONST_RU_POLY, V2, V2
// Compute the GF(2) product of the CRC polynomial in VO with T1(x) in
// V2 and XOR the intermediate result, T2(x), with the value in V1.
// The final result is in the rightmost word of V2.
VUPLLF V2, V2
VGFMAG CONST_CRC_POLY, V2, V1, V2
done:
VLGVF $2, V2, R2
XOR $0xffffffff, R2 // NOTW R2
MOVWZ R2, ret + 32(FP)
RET
crash:
MOVD $0, (R0) // input size is less than 64-bytes

284
vendor/github.com/klauspost/crc32/crc32_test.go generated vendored
View File

@@ -1,284 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package crc32
import (
crand "crypto/rand"
"hash"
mrand "math/rand"
"testing"
)
type test struct {
ieee, castagnoli uint32
in string
}
var golden = []test{
{0x0, 0x0, ""},
{0xe8b7be43, 0xc1d04330, "a"},
{0x9e83486d, 0xe2a22936, "ab"},
{0x352441c2, 0x364b3fb7, "abc"},
{0xed82cd11, 0x92c80a31, "abcd"},
{0x8587d865, 0xc450d697, "abcde"},
{0x4b8e39ef, 0x53bceff1, "abcdef"},
{0x312a6aa6, 0xe627f441, "abcdefg"},
{0xaeef2a50, 0xa9421b7, "abcdefgh"},
{0x8da988af, 0x2ddc99fc, "abcdefghi"},
{0x3981703a, 0xe6599437, "abcdefghij"},
{0x6b9cdfe7, 0xb2cc01fe, "Discard medicine more than two years old."},
{0xc90ef73f, 0xe28207f, "He who has a shady past knows that nice guys finish last."},
{0xb902341f, 0xbe93f964, "I wouldn't marry him with a ten foot pole."},
{0x42080e8, 0x9e3be0c3, "Free! Free!/A trip/to Mars/for 900/empty jars/Burma Shave"},
{0x154c6d11, 0xf505ef04, "The days of the digital watch are numbered. -Tom Stoppard"},
{0x4c418325, 0x85d3dc82, "Nepal premier won't resign."},
{0x33955150, 0xc5142380, "For every action there is an equal and opposite government program."},
{0x26216a4b, 0x75eb77dd, "His money is twice tainted: 'taint yours and 'taint mine."},
{0x1abbe45e, 0x91ebe9f7, "There is no reason for any individual to have a computer in their home. -Ken Olsen, 1977"},
{0xc89a94f7, 0xf0b1168e, "It's a tiny change to the code and not completely disgusting. - Bob Manchek"},
{0xab3abe14, 0x572b74e2, "size: a.out: bad magic"},
{0xbab102b6, 0x8a58a6d5, "The major problem is with sendmail. -Mark Horton"},
{0x999149d7, 0x9c426c50, "Give me a rock, paper and scissors and I will move the world. CCFestoon"},
{0x6d52a33c, 0x735400a4, "If the enemy is within range, then so are you."},
{0x90631e8d, 0xbec49c95, "It's well we cannot hear the screams/That we create in others' dreams."},
{0x78309130, 0xa95a2079, "You remind me of a TV show, but that's all right: I watch it anyway."},
{0x7d0a377f, 0xde2e65c5, "C is as portable as Stonehedge!!"},
{0x8c79fd79, 0x297a88ed, "Even if I could be Shakespeare, I think I should still choose to be Faraday. - A. Huxley"},
{0xa20b7167, 0x66ed1d8b, "The fugacity of a constituent in a mixture of gases at a given temperature is proportional to its mole fraction. Lewis-Randall Rule"},
{0x8e0bb443, 0xdcded527, "How can you write a big system without C++? -Paul Glick"},
}
// testGoldenIEEE verifies that the given function returns
// correct IEEE checksums.
func testGoldenIEEE(t *testing.T, crcFunc func(b []byte) uint32) {
for _, g := range golden {
if crc := crcFunc([]byte(g.in)); crc != g.ieee {
t.Errorf("IEEE(%s) = 0x%x want 0x%x", g.in, crc, g.ieee)
}
}
}
// testGoldenCastagnoli verifies that the given function returns
// correct IEEE checksums.
func testGoldenCastagnoli(t *testing.T, crcFunc func(b []byte) uint32) {
for _, g := range golden {
if crc := crcFunc([]byte(g.in)); crc != g.castagnoli {
t.Errorf("Castagnoli(%s) = 0x%x want 0x%x", g.in, crc, g.castagnoli)
}
}
}
// testCrossCheck generates random buffers of various lengths and verifies that
// the two "update" functions return the same result.
func testCrossCheck(t *testing.T, crcFunc1, crcFunc2 func(crc uint32, b []byte) uint32) {
// The AMD64 implementation has some cutoffs at lengths 168*3=504 and
// 1344*3=4032. We should make sure lengths around these values are in the
// list.
lengths := []int{0, 1, 2, 3, 4, 5, 10, 16, 50, 100, 128,
500, 501, 502, 503, 504, 505, 512, 1000, 1024, 2000,
4030, 4031, 4032, 4033, 4036, 4040, 4048, 4096, 5000, 10000}
for _, length := range lengths {
p := make([]byte, length)
_, _ = crand.Read(p)
crcInit := uint32(mrand.Int63())
crc1 := crcFunc1(crcInit, p)
crc2 := crcFunc2(crcInit, p)
if crc1 != crc2 {
t.Errorf("mismatch: 0x%x vs 0x%x (buffer length %d)", crc1, crc2, length)
}
}
}
// TestSimple tests the simple generic algorithm.
func TestSimple(t *testing.T) {
tab := simpleMakeTable(IEEE)
testGoldenIEEE(t, func(b []byte) uint32 {
return simpleUpdate(0, tab, b)
})
tab = simpleMakeTable(Castagnoli)
testGoldenCastagnoli(t, func(b []byte) uint32 {
return simpleUpdate(0, tab, b)
})
}
// TestSimple tests the slicing-by-8 algorithm.
func TestSlicing(t *testing.T) {
tab := slicingMakeTable(IEEE)
testGoldenIEEE(t, func(b []byte) uint32 {
return slicingUpdate(0, tab, b)
})
tab = slicingMakeTable(Castagnoli)
testGoldenCastagnoli(t, func(b []byte) uint32 {
return slicingUpdate(0, tab, b)
})
// Cross-check various polys against the simple algorithm.
for _, poly := range []uint32{IEEE, Castagnoli, Koopman, 0xD5828281} {
t1 := simpleMakeTable(poly)
f1 := func(crc uint32, b []byte) uint32 {
return simpleUpdate(crc, t1, b)
}
t2 := slicingMakeTable(poly)
f2 := func(crc uint32, b []byte) uint32 {
return slicingUpdate(crc, t2, b)
}
testCrossCheck(t, f1, f2)
}
}
func TestArchIEEE(t *testing.T) {
if !archAvailableIEEE() {
t.Skip("Arch-specific IEEE not available.")
}
archInitIEEE()
slicingTable := slicingMakeTable(IEEE)
testCrossCheck(t, archUpdateIEEE, func(crc uint32, b []byte) uint32 {
return slicingUpdate(crc, slicingTable, b)
})
}
func TestArchCastagnoli(t *testing.T) {
if !archAvailableCastagnoli() {
t.Skip("Arch-specific Castagnoli not available.")
}
archInitCastagnoli()
slicingTable := slicingMakeTable(Castagnoli)
testCrossCheck(t, archUpdateCastagnoli, func(crc uint32, b []byte) uint32 {
return slicingUpdate(crc, slicingTable, b)
})
}
func TestGolden(t *testing.T) {
testGoldenIEEE(t, ChecksumIEEE)
// Some implementations have special code to deal with misaligned
// data; test that as well.
for delta := 1; delta <= 7; delta++ {
testGoldenIEEE(t, func(b []byte) uint32 {
ieee := NewIEEE()
d := delta
if d >= len(b) {
d = len(b)
}
ieee.Write(b[:d])
ieee.Write(b[d:])
return ieee.Sum32()
})
}
castagnoliTab := MakeTable(Castagnoli)
if castagnoliTab == nil {
t.Errorf("nil Castagnoli Table")
}
testGoldenCastagnoli(t, func(b []byte) uint32 {
castagnoli := New(castagnoliTab)
castagnoli.Write(b)
return castagnoli.Sum32()
})
// Some implementations have special code to deal with misaligned
// data; test that as well.
for delta := 1; delta <= 7; delta++ {
testGoldenCastagnoli(t, func(b []byte) uint32 {
castagnoli := New(castagnoliTab)
d := delta
if d >= len(b) {
d = len(b)
}
castagnoli.Write(b[:d])
castagnoli.Write(b[d:])
return castagnoli.Sum32()
})
}
}
func BenchmarkIEEECrc40B(b *testing.B) {
benchmark(b, NewIEEE(), 40, 0)
}
func BenchmarkIEEECrc1KB(b *testing.B) {
benchmark(b, NewIEEE(), 1<<10, 0)
}
func BenchmarkIEEECrc4KB(b *testing.B) {
benchmark(b, NewIEEE(), 4<<10, 0)
}
func BenchmarkIEEECrc32KB(b *testing.B) {
benchmark(b, NewIEEE(), 32<<10, 0)
}
func BenchmarkCastagnoliCrc15B(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 15, 0)
}
func BenchmarkCastagnoliCrc15BMisaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 15, 1)
}
func BenchmarkCastagnoliCrc40B(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 40, 0)
}
func BenchmarkCastagnoliCrc40BMisaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 40, 1)
}
func BenchmarkCastagnoliCrc512(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 512, 0)
}
func BenchmarkCastagnoliCrc512Misaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 512, 1)
}
func BenchmarkCastagnoliCrc1KB(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 1<<10, 0)
}
func BenchmarkCastagnoliCrc1KBMisaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 1<<10, 1)
}
func BenchmarkCastagnoliCrc4KB(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 4<<10, 0)
}
func BenchmarkCastagnoliCrc4KBMisaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 4<<10, 1)
}
func BenchmarkCastagnoliCrc32KB(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 32<<10, 0)
}
func BenchmarkCastagnoliCrc32KBMisaligned(b *testing.B) {
benchmark(b, New(MakeTable(Castagnoli)), 32<<10, 1)
}
func benchmark(b *testing.B, h hash.Hash32, n, alignment int64) {
b.SetBytes(n)
data := make([]byte, n+alignment)
data = data[alignment:]
for i := range data {
data[i] = byte(i)
}
in := make([]byte, 0, h.Size())
// Warm up
h.Reset()
h.Write(data)
h.Sum(in)
b.ResetTimer()
for i := 0; i < b.N; i++ {
h.Reset()
h.Write(data)
h.Sum(in)
}
}

28
vendor/github.com/klauspost/crc32/example_test.go generated vendored
View File

@@ -1,28 +0,0 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package crc32_test
import (
"fmt"
"hash/crc32"
)
func ExampleMakeTable() {
// In this package, the CRC polynomial is represented in reversed notation,
// or LSB-first representation.
//
// LSB-first representation is a hexadecimal number with n bits, in which the
// most significant bit represents the coefficient of x⁰ and the least significant
// bit represents the coefficient of xⁿ⁻¹ (the coefficient for xⁿ is implicit).
//
// For example, CRC32-Q, as defined by the following polynomial,
// x³²+ x³¹+ x²⁴+ x²²+ x¹⁶+ x¹⁴+ x⁸+ x⁷+ x⁵+ x³+ x¹+ x⁰
// has the reversed notation 0b11010101100000101000001010000001, so the value
// that should be passed to MakeTable is 0xD5828281.
crc32q := crc32.MakeTable(0xD5828281)
fmt.Printf("%08x\n", crc32.Checksum([]byte("Hello world"), crc32q))
// Output:
// 2964d064
}

209
vendor/github.com/klauspost/reedsolomon/examples_test.go generated vendored
View File

@@ -1,209 +0,0 @@
package reedsolomon_test
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"math/rand"
"github.com/klauspost/reedsolomon"
)
func fillRandom(p []byte) {
for i := 0; i < len(p); i += 7 {
val := rand.Int63()
for j := 0; i+j < len(p) && j < 7; j++ {
p[i+j] = byte(val)
val >>= 8
}
}
}
// Simple example of how to use all functions of the Encoder.
// Note that all error checks have been removed to keep it short.
func ExampleEncoder() {
// Create some sample data
var data = make([]byte, 250000)
fillRandom(data)
// Create an encoder with 17 data and 3 parity slices.
enc, _ := reedsolomon.New(17, 3)
// Split the data into shards
shards, _ := enc.Split(data)
// Encode the parity set
_ = enc.Encode(shards)
// Verify the parity set
ok, _ := enc.Verify(shards)
if ok {
fmt.Println("ok")
}
// Delete two shards
shards[10], shards[11] = nil, nil
// Reconstruct the shards
_ = enc.Reconstruct(shards)
// Verify the data set
ok, _ = enc.Verify(shards)
if ok {
fmt.Println("ok")
}
// Output: ok
// ok
}
// This demonstrates that shards can be arbitrary sliced and
// merged and still remain valid.
func ExampleEncoder_slicing() {
// Create some sample data
var data = make([]byte, 250000)
fillRandom(data)
// Create 5 data slices of 50000 elements each
enc, _ := reedsolomon.New(5, 3)
shards, _ := enc.Split(data)
err := enc.Encode(shards)
if err != nil {
panic(err)
}
// Check that it verifies
ok, err := enc.Verify(shards)
if ok && err == nil {
fmt.Println("encode ok")
}
// Split the data set of 50000 elements into two of 25000
splitA := make([][]byte, 8)
splitB := make([][]byte, 8)
// Merge into a 100000 element set
merged := make([][]byte, 8)
// Split/merge the shards
for i := range shards {
splitA[i] = shards[i][:25000]
splitB[i] = shards[i][25000:]
// Concencate it to itself
merged[i] = append(make([]byte, 0, len(shards[i])*2), shards[i]...)
merged[i] = append(merged[i], shards[i]...)
}
// Each part should still verify as ok.
ok, err = enc.Verify(shards)
if ok && err == nil {
fmt.Println("splitA ok")
}
ok, err = enc.Verify(splitB)
if ok && err == nil {
fmt.Println("splitB ok")
}
ok, err = enc.Verify(merged)
if ok && err == nil {
fmt.Println("merge ok")
}
// Output: encode ok
// splitA ok
// splitB ok
// merge ok
}
// This demonstrates that shards can xor'ed and
// still remain a valid set.
//
// The xor value must be the same for element 'n' in each shard,
// except if you xor with a similar sized encoded shard set.
func ExampleEncoder_xor() {
// Create some sample data
var data = make([]byte, 25000)
fillRandom(data)
// Create 5 data slices of 5000 elements each
enc, _ := reedsolomon.New(5, 3)
shards, _ := enc.Split(data)
err := enc.Encode(shards)
if err != nil {
panic(err)
}
// Check that it verifies
ok, err := enc.Verify(shards)
if !ok || err != nil {
fmt.Println("falied initial verify", err)
}
// Create an xor'ed set
xored := make([][]byte, 8)
// We xor by the index, so you can see that the xor can change,
// It should however be constant vertically through your slices.
for i := range shards {
xored[i] = make([]byte, len(shards[i]))
for j := range xored[i] {
xored[i][j] = shards[i][j] ^ byte(j&0xff)
}
}
// Each part should still verify as ok.
ok, err = enc.Verify(xored)
if ok && err == nil {
fmt.Println("verified ok after xor")
}
// Output: verified ok after xor
}
// This will show a simple stream encoder where we encode from
// a []io.Reader which contain a reader for each shard.
//
// Input and output can be exchanged with files, network streams
// or what may suit your needs.
func ExampleStreamEncoder() {
dataShards := 5
parityShards := 2
// Create a StreamEncoder with the number of data and
// parity shards.
rs, err := reedsolomon.NewStream(dataShards, parityShards)
if err != nil {
log.Fatal(err)
}
shardSize := 50000
// Create input data shards.
input := make([][]byte, dataShards)
for s := range input {
input[s] = make([]byte, shardSize)
fillRandom(input[s])
}
// Convert our buffers to io.Readers
readers := make([]io.Reader, dataShards)
for i := range readers {
readers[i] = io.Reader(bytes.NewBuffer(input[i]))
}
// Create our output io.Writers
out := make([]io.Writer, parityShards)
for i := range out {
out[i] = ioutil.Discard
}
// Encode from input to output.
err = rs.Encode(readers, out)
if err != nil {
log.Fatal(err)
}
fmt.Println("ok")
// OUTPUT: ok
}

155
vendor/github.com/klauspost/reedsolomon/galois_test.go generated vendored
View File

@@ -1,155 +0,0 @@
/**
* Unit tests for Galois
*
* Copyright 2015, Klaus Post
* Copyright 2015, Backblaze, Inc.
*/
package reedsolomon
import (
"bytes"
"testing"
)
func TestAssociativity(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
for j := 0; j < 256; j++ {
b := byte(j)
for k := 0; k < 256; k++ {
c := byte(k)
x := galAdd(a, galAdd(b, c))
y := galAdd(galAdd(a, b), c)
if x != y {
t.Fatal("add does not match:", x, "!=", y)
}
x = galMultiply(a, galMultiply(b, c))
y = galMultiply(galMultiply(a, b), c)
if x != y {
t.Fatal("multiply does not match:", x, "!=", y)
}
}
}
}
}
func TestIdentity(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
b := galAdd(a, 0)
if a != b {
t.Fatal("Add zero should yield same result", a, "!=", b)
}
b = galMultiply(a, 1)
if a != b {
t.Fatal("Mul by one should yield same result", a, "!=", b)
}
}
}
func TestInverse(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
b := galSub(0, a)
c := galAdd(a, b)
if c != 0 {
t.Fatal("inverse sub/add", c, "!=", 0)
}
if a != 0 {
b = galDivide(1, a)
c = galMultiply(a, b)
if c != 1 {
t.Fatal("inverse div/mul", c, "!=", 1)
}
}
}
}
func TestCommutativity(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
for j := 0; j < 256; j++ {
b := byte(j)
x := galAdd(a, b)
y := galAdd(b, a)
if x != y {
t.Fatal(x, "!= ", y)
}
x = galMultiply(a, b)
y = galMultiply(b, a)
if x != y {
t.Fatal(x, "!= ", y)
}
}
}
}
func TestDistributivity(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
for j := 0; j < 256; j++ {
b := byte(j)
for k := 0; k < 256; k++ {
c := byte(k)
x := galMultiply(a, galAdd(b, c))
y := galAdd(galMultiply(a, b), galMultiply(a, c))
if x != y {
t.Fatal(x, "!= ", y)
}
}
}
}
}
func TestExp(t *testing.T) {
for i := 0; i < 256; i++ {
a := byte(i)
power := byte(1)
for j := 0; j < 256; j++ {
x := galExp(a, j)
if x != power {
t.Fatal(x, "!=", power)
}
power = galMultiply(power, a)
}
}
}
func TestGalois(t *testing.T) {
// These values were copied output of the Python code.
if galMultiply(3, 4) != 12 {
t.Fatal("galMultiply(3, 4) != 12")
}
if galMultiply(7, 7) != 21 {
t.Fatal("galMultiply(7, 7) != 21")
}
if galMultiply(23, 45) != 41 {
t.Fatal("galMultiply(23, 45) != 41")
}
// Test slices (>16 entries to test assembler)
in := []byte{0, 1, 2, 3, 4, 5, 6, 10, 50, 100, 150, 174, 201, 255, 99, 32, 67, 85}
out := make([]byte, len(in))
galMulSlice(25, in, out, false, false)
expect := []byte{0x0, 0x19, 0x32, 0x2b, 0x64, 0x7d, 0x56, 0xfa, 0xb8, 0x6d, 0xc7, 0x85, 0xc3, 0x1f, 0x22, 0x7, 0x25, 0xfe}
if 0 != bytes.Compare(out, expect) {
t.Errorf("got %#v, expected %#v", out, expect)
}
galMulSlice(177, in, out, false, false)
expect = []byte{0x0, 0xb1, 0x7f, 0xce, 0xfe, 0x4f, 0x81, 0x9e, 0x3, 0x6, 0xe8, 0x75, 0xbd, 0x40, 0x36, 0xa3, 0x95, 0xcb}
if 0 != bytes.Compare(out, expect) {
t.Errorf("got %#v, expected %#v", out, expect)
}
if galExp(2, 2) != 4 {
t.Fatal("galExp(2, 2) != 4")
}
if galExp(5, 20) != 235 {
t.Fatal("galExp(5, 20) != 235")
}
if galExp(13, 7) != 43 {
t.Fatal("galExp(13, 7) != 43")
}
}

125
vendor/github.com/klauspost/reedsolomon/inversion_tree_test.go generated vendored
View File

@@ -1,125 +0,0 @@
/**
* Unit tests for inversion tree.
*
* Copyright 2016, Peter Collins
*/
package reedsolomon
import (
"testing"
)
func TestNewInversionTree(t *testing.T) {
tree := newInversionTree(3, 2)
children := len(tree.root.children)
if children != 5 {
t.Fatal("Root node children list length", children, "!=", 5)
}
str := tree.root.matrix.String()
expect := "[[1, 0, 0], [0, 1, 0], [0, 0, 1]]"
if str != expect {
t.Fatal(str, "!=", expect)
}
}
func TestGetInvertedMatrix(t *testing.T) {
tree := newInversionTree(3, 2)
matrix := tree.GetInvertedMatrix([]int{})
str := matrix.String()
expect := "[[1, 0, 0], [0, 1, 0], [0, 0, 1]]"
if str != expect {
t.Fatal(str, "!=", expect)
}
matrix = tree.GetInvertedMatrix([]int{1})
if matrix != nil {
t.Fatal(matrix, "!= nil")
}
matrix = tree.GetInvertedMatrix([]int{1, 2})
if matrix != nil {
t.Fatal(matrix, "!= nil")
}
matrix, err := newMatrix(3, 3)
if err != nil {
t.Fatalf("Failed initializing new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{1}, matrix, 5)
if err != nil {
t.Fatalf("Failed inserting new Matrix : %s", err)
}
cachedMatrix := tree.GetInvertedMatrix([]int{1})
if cachedMatrix == nil {
t.Fatal(cachedMatrix, "== nil")
}
if matrix.String() != cachedMatrix.String() {
t.Fatal(matrix.String(), "!=", cachedMatrix.String())
}
}
func TestInsertInvertedMatrix(t *testing.T) {
tree := newInversionTree(3, 2)
matrix, err := newMatrix(3, 3)
if err != nil {
t.Fatalf("Failed initializing new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{1}, matrix, 5)
if err != nil {
t.Fatalf("Failed inserting new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{}, matrix, 5)
if err == nil {
t.Fatal("Should have failed inserting the root node matrix", matrix)
}
matrix, err = newMatrix(3, 2)
if err != nil {
t.Fatalf("Failed initializing new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{2}, matrix, 5)
if err == nil {
t.Fatal("Should have failed inserting a non-square matrix", matrix)
}
matrix, err = newMatrix(3, 3)
if err != nil {
t.Fatalf("Failed initializing new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{0, 1}, matrix, 5)
if err != nil {
t.Fatalf("Failed inserting new Matrix : %s", err)
}
}
func TestDoubleInsertInvertedMatrix(t *testing.T) {
tree := newInversionTree(3, 2)
matrix, err := newMatrix(3, 3)
if err != nil {
t.Fatalf("Failed initializing new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{1}, matrix, 5)
if err != nil {
t.Fatalf("Failed inserting new Matrix : %s", err)
}
err = tree.InsertInvertedMatrix([]int{1}, matrix, 5)
if err != nil {
t.Fatalf("Failed inserting new Matrix : %s", err)
}
cachedMatrix := tree.GetInvertedMatrix([]int{1})
if cachedMatrix == nil {
t.Fatal(cachedMatrix, "== nil")
}
if matrix.String() != cachedMatrix.String() {
t.Fatal(matrix.String(), "!=", cachedMatrix.String())
}
}

217
vendor/github.com/klauspost/reedsolomon/matrix_test.go generated vendored
View File

@@ -1,217 +0,0 @@
/**
* Unit tests for Matrix
*
* Copyright 2015, Klaus Post
* Copyright 2015, Backblaze, Inc. All rights reserved.
*/
package reedsolomon
import (
"testing"
)
// TestNewMatrix - Tests validate the result for invalid input and the allocations made by newMatrix method.
func TestNewMatrix(t *testing.T) {
testCases := []struct {
rows int
columns int
// flag to indicate whether the test should pass.
shouldPass bool
expectedResult matrix
expectedErr error
}{
// Test case - 1.
// Test case with a negative row size.
{-1, 10, false, nil, errInvalidRowSize},
// Test case - 2.
// Test case with a negative column size.
{10, -1, false, nil, errInvalidColSize},
// Test case - 3.
// Test case with negative value for both row and column size.
{-1, -1, false, nil, errInvalidRowSize},
// Test case - 4.
// Test case with 0 value for row size.
{0, 10, false, nil, errInvalidRowSize},
// Test case - 5.
// Test case with 0 value for column size.
{-1, 0, false, nil, errInvalidRowSize},
// Test case - 6.
// Test case with 0 value for both row and column size.
{0, 0, false, nil, errInvalidRowSize},
}
for i, testCase := range testCases {
actualResult, actualErr := newMatrix(testCase.rows, testCase.columns)
if actualErr != nil && testCase.shouldPass {
t.Errorf("Test %d: Expected to pass, but failed with: <ERROR> %s", i+1, actualErr.Error())
}
if actualErr == nil && !testCase.shouldPass {
t.Errorf("Test %d: Expected to fail with <ERROR> \"%s\", but passed instead.", i+1, testCase.expectedErr)
}
// Failed as expected, but does it fail for the expected reason.
if actualErr != nil && !testCase.shouldPass {
if testCase.expectedErr != actualErr {
t.Errorf("Test %d: Expected to fail with error \"%s\", but instead failed with error \"%s\" instead.", i+1, testCase.expectedErr, actualErr)
}
}
// Test passes as expected, but the output values
// are verified for correctness here.
if actualErr == nil && testCase.shouldPass {
if testCase.rows != len(actualResult) {
// End the tests here if the the size doesn't match number of rows.
t.Fatalf("Test %d: Expected the size of the row of the new matrix to be `%d`, but instead found `%d`", i+1, testCase.rows, len(actualResult))
}
// Iterating over each row and validating the size of the column.
for j, row := range actualResult {
// If the row check passes, verify the size of each columns.
if testCase.columns != len(row) {
t.Errorf("Test %d: Row %d: Expected the size of the column of the new matrix to be `%d`, but instead found `%d`", i+1, j+1, testCase.columns, len(row))
}
}
}
}
}
// TestMatrixIdentity - validates the method for returning identity matrix of given size.
func TestMatrixIdentity(t *testing.T) {
m, err := identityMatrix(3)
if err != nil {
t.Fatal(err)
}
str := m.String()
expect := "[[1, 0, 0], [0, 1, 0], [0, 0, 1]]"
if str != expect {
t.Fatal(str, "!=", expect)
}
}
// Tests validate the output of matix multiplication method.
func TestMatrixMultiply(t *testing.T) {
m1, err := newMatrixData(
[][]byte{
[]byte{1, 2},
[]byte{3, 4},
})
if err != nil {
t.Fatal(err)
}
m2, err := newMatrixData(
[][]byte{
[]byte{5, 6},
[]byte{7, 8},
})
if err != nil {
t.Fatal(err)
}
actual, err := m1.Multiply(m2)
if err != nil {
t.Fatal(err)
}
str := actual.String()
expect := "[[11, 22], [19, 42]]"
if str != expect {
t.Fatal(str, "!=", expect)
}
}
// Tests validate the output of the method with computes inverse of matrix.
func TestMatrixInverse(t *testing.T) {
testCases := []struct {
matrixData [][]byte
// expected inverse matrix.
expectedResult string
// flag indicating whether the test should pass.
shouldPass bool
expectedErr error
}{
// Test case - 1.
// Test case validating inverse of the input Matrix.
{
// input data to construct the matrix.
[][]byte{
[]byte{56, 23, 98},
[]byte{3, 100, 200},
[]byte{45, 201, 123},
},
// expected Inverse matrix.
"[[175, 133, 33], [130, 13, 245], [112, 35, 126]]",
// test is expected to pass.
true,
nil,
},
// Test case - 2.
// Test case validating inverse of the input Matrix.
{
// input data to contruct the matrix.
[][]byte{
[]byte{1, 0, 0, 0, 0},
[]byte{0, 1, 0, 0, 0},
[]byte{0, 0, 0, 1, 0},
[]byte{0, 0, 0, 0, 1},
[]byte{7, 7, 6, 6, 1},
},
// expectedInverse matrix.
"[[1, 0, 0, 0, 0]," +
" [0, 1, 0, 0, 0]," +
" [123, 123, 1, 122, 122]," +
" [0, 0, 1, 0, 0]," +
" [0, 0, 0, 1, 0]]",
// test is expected to pass.
true,
nil,
},
// Test case with a non-square matrix.
// expected to fail with errNotSquare.
{
[][]byte{
[]byte{56, 23},
[]byte{3, 100},
[]byte{45, 201},
},
"",
false,
errNotSquare,
},
// Test case with singular matrix.
// expected to fail with error errSingular.
{
[][]byte{
[]byte{4, 2},
[]byte{12, 6},
},
"",
false,
errSingular,
},
}
for i, testCase := range testCases {
m, err := newMatrixData(testCase.matrixData)
if err != nil {
t.Fatalf("Test %d: Failed initializing new Matrix : %s", i+1, err)
}
actualResult, actualErr := m.Invert()
if actualErr != nil && testCase.shouldPass {
t.Errorf("Test %d: Expected to pass, but failed with: <ERROR> %s", i+1, actualErr.Error())
}
if actualErr == nil && !testCase.shouldPass {
t.Errorf("Test %d: Expected to fail with <ERROR> \"%s\", but passed instead.", i+1, testCase.expectedErr)
}
// Failed as expected, but does it fail for the expected reason.
if actualErr != nil && !testCase.shouldPass {
if testCase.expectedErr != actualErr {
t.Errorf("Test %d: Expected to fail with error \"%s\", but instead failed with error \"%s\" instead.", i+1, testCase.expectedErr, actualErr)
}
}
// Test passes as expected, but the output values
// are verified for correctness here.
if actualErr == nil && testCase.shouldPass {
if testCase.expectedResult != actualResult.String() {
t.Errorf("Test %d: The inverse matrix doesnt't match the expected result", i+1)
}
}
}
}

761
vendor/github.com/klauspost/reedsolomon/reedsolomon_test.go generated vendored
View File

@@ -1,761 +0,0 @@
/**
* Unit tests for ReedSolomon
*
* Copyright 2015, Klaus Post
* Copyright 2015, Backblaze, Inc. All rights reserved.
*/
package reedsolomon
import (
"bytes"
"math/rand"
"runtime"
"testing"
)
func testOpts() [][]Option {
if !testing.Short() {
return [][]Option{}
}
opts := [][]Option{
{WithMaxGoroutines(1), WithMinSplitSize(500), withSSE3(false), withAVX2(false)},
{WithMaxGoroutines(5000), WithMinSplitSize(50), withSSE3(false), withAVX2(false)},
{WithMaxGoroutines(5000), WithMinSplitSize(500000), withSSE3(false), withAVX2(false)},
{WithMaxGoroutines(1), WithMinSplitSize(500000), withSSE3(false), withAVX2(false)},
}
for _, o := range opts[:] {
if defaultOptions.useSSSE3 {
n := make([]Option, len(o), len(o)+1)
copy(n, o)
n = append(n, withSSE3(true))
opts = append(opts, n)
}
if defaultOptions.useAVX2 {
n := make([]Option, len(o), len(o)+1)
copy(n, o)
n = append(n, withAVX2(true))
opts = append(opts, n)
}
}
return opts
}
func TestEncoding(t *testing.T) {
testEncoding(t)
for _, o := range testOpts() {
testEncoding(t, o...)
}
}
func testEncoding(t *testing.T, o ...Option) {
perShard := 50000
r, err := New(10, 3, o...)
if err != nil {
t.Fatal(err)
}
shards := make([][]byte, 13)
for s := range shards {
shards[s] = make([]byte, perShard)
}
rand.Seed(0)
for s := 0; s < 13; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
t.Fatal(err)
}
ok, err := r.Verify(shards)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
err = r.Encode(make([][]byte, 1))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
badShards := make([][]byte, 13)
badShards[0] = make([]byte, 1)
err = r.Encode(badShards)
if err != ErrShardSize {
t.Errorf("expected %v, got %v", ErrShardSize, err)
}
}
func TestReconstruct(t *testing.T) {
testReconstruct(t)
for _, o := range testOpts() {
testReconstruct(t, o...)
}
}
func testReconstruct(t *testing.T, o ...Option) {
perShard := 50000
r, err := New(10, 3, o...)
if err != nil {
t.Fatal(err)
}
shards := make([][]byte, 13)
for s := range shards {
shards[s] = make([]byte, perShard)
}
rand.Seed(0)
for s := 0; s < 13; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
t.Fatal(err)
}
// Reconstruct with all shards present
err = r.Reconstruct(shards)
if err != nil {
t.Fatal(err)
}
// Reconstruct with 10 shards present
shards[0] = nil
shards[7] = nil
shards[11] = nil
err = r.Reconstruct(shards)
if err != nil {
t.Fatal(err)
}
ok, err := r.Verify(shards)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
// Reconstruct with 9 shards present (should fail)
shards[0] = nil
shards[4] = nil
shards[7] = nil
shards[11] = nil
err = r.Reconstruct(shards)
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Reconstruct(make([][]byte, 1))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Reconstruct(make([][]byte, 13))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
}
func TestVerify(t *testing.T) {
testVerify(t)
for _, o := range testOpts() {
testVerify(t, o...)
}
}
func testVerify(t *testing.T, o ...Option) {
perShard := 33333
r, err := New(10, 4, o...)
if err != nil {
t.Fatal(err)
}
shards := make([][]byte, 14)
for s := range shards {
shards[s] = make([]byte, perShard)
}
rand.Seed(0)
for s := 0; s < 10; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
t.Fatal(err)
}
ok, err := r.Verify(shards)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
// Put in random data. Verification should fail
fillRandom(shards[10])
ok, err = r.Verify(shards)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("Verification did not fail")
}
// Re-encode
err = r.Encode(shards)
if err != nil {
t.Fatal(err)
}
// Fill a data segment with random data
fillRandom(shards[0])
ok, err = r.Verify(shards)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("Verification did not fail")
}
_, err = r.Verify(make([][]byte, 1))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
_, err = r.Verify(make([][]byte, 14))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
}
func TestOneEncode(t *testing.T) {
codec, err := New(5, 5)
if err != nil {
t.Fatal(err)
}
shards := [][]byte{
{0, 1},
{4, 5},
{2, 3},
{6, 7},
{8, 9},
{0, 0},
{0, 0},
{0, 0},
{0, 0},
{0, 0},
}
codec.Encode(shards)
if shards[5][0] != 12 || shards[5][1] != 13 {
t.Fatal("shard 5 mismatch")
}
if shards[6][0] != 10 || shards[6][1] != 11 {
t.Fatal("shard 6 mismatch")
}
if shards[7][0] != 14 || shards[7][1] != 15 {
t.Fatal("shard 7 mismatch")
}
if shards[8][0] != 90 || shards[8][1] != 91 {
t.Fatal("shard 8 mismatch")
}
if shards[9][0] != 94 || shards[9][1] != 95 {
t.Fatal("shard 9 mismatch")
}
ok, err := codec.Verify(shards)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("did not verify")
}
shards[8][0]++
ok, err = codec.Verify(shards)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("verify did not fail as expected")
}
}
func fillRandom(p []byte) {
for i := 0; i < len(p); i += 7 {
val := rand.Int63()
for j := 0; i+j < len(p) && j < 7; j++ {
p[i+j] = byte(val)
val >>= 8
}
}
}
func benchmarkEncode(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := New(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
shards := make([][]byte, dataShards+parityShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
b.SetBytes(int64(shardSize * dataShards))
b.ResetTimer()
for i := 0; i < b.N; i++ {
err = r.Encode(shards)
if err != nil {
b.Fatal(err)
}
}
}
func BenchmarkEncode10x2x10000(b *testing.B) {
benchmarkEncode(b, 10, 2, 10000)
}
func BenchmarkEncode100x20x10000(b *testing.B) {
benchmarkEncode(b, 100, 20, 10000)
}
func BenchmarkEncode17x3x1M(b *testing.B) {
benchmarkEncode(b, 17, 3, 1024*1024)
}
// Benchmark 10 data shards and 4 parity shards with 16MB each.
func BenchmarkEncode10x4x16M(b *testing.B) {
benchmarkEncode(b, 10, 4, 16*1024*1024)
}
// Benchmark 5 data shards and 2 parity shards with 1MB each.
func BenchmarkEncode5x2x1M(b *testing.B) {
benchmarkEncode(b, 5, 2, 1024*1024)
}
// Benchmark 1 data shards and 2 parity shards with 1MB each.
func BenchmarkEncode10x2x1M(b *testing.B) {
benchmarkEncode(b, 10, 2, 1024*1024)
}
// Benchmark 10 data shards and 4 parity shards with 1MB each.
func BenchmarkEncode10x4x1M(b *testing.B) {
benchmarkEncode(b, 10, 4, 1024*1024)
}
// Benchmark 50 data shards and 20 parity shards with 1MB each.
func BenchmarkEncode50x20x1M(b *testing.B) {
benchmarkEncode(b, 50, 20, 1024*1024)
}
// Benchmark 17 data shards and 3 parity shards with 16MB each.
func BenchmarkEncode17x3x16M(b *testing.B) {
benchmarkEncode(b, 17, 3, 16*1024*1024)
}
func benchmarkVerify(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := New(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
shards := make([][]byte, parityShards+dataShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
b.Fatal(err)
}
b.SetBytes(int64(shardSize * dataShards))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err = r.Verify(shards)
if err != nil {
b.Fatal(err)
}
}
}
// Benchmark 10 data slices with 2 parity slices holding 10000 bytes each
func BenchmarkVerify10x2x10000(b *testing.B) {
benchmarkVerify(b, 10, 2, 10000)
}
// Benchmark 50 data slices with 5 parity slices holding 100000 bytes each
func BenchmarkVerify50x5x50000(b *testing.B) {
benchmarkVerify(b, 50, 5, 100000)
}
// Benchmark 10 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkVerify10x2x1M(b *testing.B) {
benchmarkVerify(b, 10, 2, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkVerify5x2x1M(b *testing.B) {
benchmarkVerify(b, 5, 2, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 1MB bytes each
func BenchmarkVerify10x4x1M(b *testing.B) {
benchmarkVerify(b, 10, 4, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkVerify50x20x1M(b *testing.B) {
benchmarkVerify(b, 50, 20, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 16MB bytes each
func BenchmarkVerify10x4x16M(b *testing.B) {
benchmarkVerify(b, 10, 4, 16*1024*1024)
}
func corruptRandom(shards [][]byte, dataShards, parityShards int) {
shardsToCorrupt := rand.Intn(parityShards)
for i := 1; i <= shardsToCorrupt; i++ {
shards[rand.Intn(dataShards+parityShards)] = nil
}
}
func benchmarkReconstruct(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := New(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
shards := make([][]byte, parityShards+dataShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
b.Fatal(err)
}
b.SetBytes(int64(shardSize * dataShards))
b.ResetTimer()
for i := 0; i < b.N; i++ {
corruptRandom(shards, dataShards, parityShards)
err = r.Reconstruct(shards)
if err != nil {
b.Fatal(err)
}
ok, err := r.Verify(shards)
if err != nil {
b.Fatal(err)
}
if !ok {
b.Fatal("Verification failed")
}
}
}
// Benchmark 10 data slices with 2 parity slices holding 10000 bytes each
func BenchmarkReconstruct10x2x10000(b *testing.B) {
benchmarkReconstruct(b, 10, 2, 10000)
}
// Benchmark 50 data slices with 5 parity slices holding 100000 bytes each
func BenchmarkReconstruct50x5x50000(b *testing.B) {
benchmarkReconstruct(b, 50, 5, 100000)
}
// Benchmark 10 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstruct10x2x1M(b *testing.B) {
benchmarkReconstruct(b, 10, 2, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstruct5x2x1M(b *testing.B) {
benchmarkReconstruct(b, 5, 2, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 1MB bytes each
func BenchmarkReconstruct10x4x1M(b *testing.B) {
benchmarkReconstruct(b, 10, 4, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstruct50x20x1M(b *testing.B) {
benchmarkReconstruct(b, 50, 20, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 16MB bytes each
func BenchmarkReconstruct10x4x16M(b *testing.B) {
benchmarkReconstruct(b, 10, 4, 16*1024*1024)
}
func benchmarkReconstructP(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := New(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
b.SetBytes(int64(shardSize * dataShards))
runtime.GOMAXPROCS(runtime.NumCPU())
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
shards := make([][]byte, parityShards+dataShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
err = r.Encode(shards)
if err != nil {
b.Fatal(err)
}
for pb.Next() {
corruptRandom(shards, dataShards, parityShards)
err = r.Reconstruct(shards)
if err != nil {
b.Fatal(err)
}
ok, err := r.Verify(shards)
if err != nil {
b.Fatal(err)
}
if !ok {
b.Fatal("Verification failed")
}
}
})
}
// Benchmark 10 data slices with 2 parity slices holding 10000 bytes each
func BenchmarkReconstructP10x2x10000(b *testing.B) {
benchmarkReconstructP(b, 10, 2, 10000)
}
// Benchmark 50 data slices with 5 parity slices holding 100000 bytes each
func BenchmarkReconstructP50x5x50000(b *testing.B) {
benchmarkReconstructP(b, 50, 5, 100000)
}
// Benchmark 10 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstructP10x2x1M(b *testing.B) {
benchmarkReconstructP(b, 10, 2, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstructP5x2x1M(b *testing.B) {
benchmarkReconstructP(b, 5, 2, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 1MB bytes each
func BenchmarkReconstructP10x4x1M(b *testing.B) {
benchmarkReconstructP(b, 10, 4, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkReconstructP50x20x1M(b *testing.B) {
benchmarkReconstructP(b, 50, 20, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 16MB bytes each
func BenchmarkReconstructP10x4x16M(b *testing.B) {
benchmarkReconstructP(b, 10, 4, 16*1024*1024)
}
func TestEncoderReconstruct(t *testing.T) {
testEncoderReconstruct(t)
for _, o := range testOpts() {
testEncoderReconstruct(t, o...)
}
}
func testEncoderReconstruct(t *testing.T, o ...Option) {
// Create some sample data
var data = make([]byte, 250000)
fillRandom(data)
// Create 5 data slices of 50000 elements each
enc, err := New(5, 3, o...)
if err != nil {
t.Fatal(err)
}
shards, err := enc.Split(data)
if err != nil {
t.Fatal(err)
}
err = enc.Encode(shards)
if err != nil {
t.Fatal(err)
}
// Check that it verifies
ok, err := enc.Verify(shards)
if !ok || err != nil {
t.Fatal("not ok:", ok, "err:", err)
}
// Delete a shard
shards[0] = nil
// Should reconstruct
err = enc.Reconstruct(shards)
if err != nil {
t.Fatal(err)
}
// Check that it verifies
ok, err = enc.Verify(shards)
if !ok || err != nil {
t.Fatal("not ok:", ok, "err:", err)
}
// Recover original bytes
buf := new(bytes.Buffer)
err = enc.Join(buf, shards, len(data))
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(buf.Bytes(), data) {
t.Fatal("recovered bytes do not match")
}
// Corrupt a shard
shards[0] = nil
shards[1][0], shards[1][500] = 75, 75
// Should reconstruct (but with corrupted data)
err = enc.Reconstruct(shards)
if err != nil {
t.Fatal(err)
}
// Check that it verifies
ok, err = enc.Verify(shards)
if ok || err != nil {
t.Fatal("error or ok:", ok, "err:", err)
}
// Recovered data should not match original
buf.Reset()
err = enc.Join(buf, shards, len(data))
if err != nil {
t.Fatal(err)
}
if bytes.Equal(buf.Bytes(), data) {
t.Fatal("corrupted data matches original")
}
}
func TestSplitJoin(t *testing.T) {
var data = make([]byte, 250000)
rand.Seed(0)
fillRandom(data)
enc, _ := New(5, 3)
shards, err := enc.Split(data)
if err != nil {
t.Fatal(err)
}
_, err = enc.Split([]byte{})
if err != ErrShortData {
t.Errorf("expected %v, got %v", ErrShortData, err)
}
buf := new(bytes.Buffer)
err = enc.Join(buf, shards, 50)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(buf.Bytes(), data[:50]) {
t.Fatal("recovered data does match original")
}
err = enc.Join(buf, [][]byte{}, 0)
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = enc.Join(buf, shards, len(data)+1)
if err != ErrShortData {
t.Errorf("expected %v, got %v", ErrShortData, err)
}
shards[0] = nil
err = enc.Join(buf, shards, len(data))
if err != ErrReconstructRequired {
t.Errorf("expected %v, got %v", ErrReconstructRequired, err)
}
}
func TestCodeSomeShards(t *testing.T) {
var data = make([]byte, 250000)
fillRandom(data)
enc, _ := New(5, 3)
r := enc.(*reedSolomon) // need to access private methods
shards, _ := enc.Split(data)
old := runtime.GOMAXPROCS(1)
r.codeSomeShards(r.parity, shards[:r.DataShards], shards[r.DataShards:], r.ParityShards, len(shards[0]))
// hopefully more than 1 CPU
runtime.GOMAXPROCS(runtime.NumCPU())
r.codeSomeShards(r.parity, shards[:r.DataShards], shards[r.DataShards:], r.ParityShards, len(shards[0]))
// reset MAXPROCS, otherwise testing complains
runtime.GOMAXPROCS(old)
}
func TestAllMatrices(t *testing.T) {
t.Skip("Skipping slow matrix check")
for i := 1; i < 257; i++ {
_, err := New(i, i)
if err != nil {
t.Fatal("creating matrix size", i, i, ":", err)
}
}
}
func TestNew(t *testing.T) {
tests := []struct {
data, parity int
err error
}{
{127, 127, nil},
{256, 256, ErrMaxShardNum},
{0, 1, ErrInvShardNum},
{1, 0, ErrInvShardNum},
{257, 1, ErrMaxShardNum},
// overflow causes r.Shards to be negative
{256, int(^uint(0) >> 1), errInvalidRowSize},
}
for _, test := range tests {
_, err := New(test.data, test.parity)
if err != test.err {
t.Errorf("New(%v, %v): expected %v, got %v", test.data, test.parity, test.err, err)
}
}
}

604
vendor/github.com/klauspost/reedsolomon/streaming_test.go generated vendored
View File

@@ -1,604 +0,0 @@
/**
* Unit tests for ReedSolomon Streaming API
*
* Copyright 2015, Klaus Post
*/
package reedsolomon
import (
"bytes"
"io"
"io/ioutil"
"math/rand"
"testing"
)
func TestStreamEncoding(t *testing.T) {
perShard := 10 << 20
if testing.Short() {
perShard = 50000
}
r, err := NewStream(10, 3)
if err != nil {
t.Fatal(err)
}
rand.Seed(0)
input := randomBytes(10, perShard)
data := toBuffers(input)
par := emptyBuffers(3)
err = r.Encode(toReaders(data), toWriters(par))
if err != nil {
t.Fatal(err)
}
// Reset Data
data = toBuffers(input)
all := append(toReaders(data), toReaders(par)...)
ok, err := r.Verify(all)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
err = r.Encode(toReaders(emptyBuffers(1)), toWriters(emptyBuffers(1)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Encode(toReaders(emptyBuffers(10)), toWriters(emptyBuffers(1)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Encode(toReaders(emptyBuffers(10)), toWriters(emptyBuffers(3)))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
badShards := emptyBuffers(10)
badShards[0] = randomBuffer(123)
err = r.Encode(toReaders(badShards), toWriters(emptyBuffers(3)))
if err != ErrShardSize {
t.Errorf("expected %v, got %v", ErrShardSize, err)
}
}
func TestStreamEncodingConcurrent(t *testing.T) {
perShard := 10 << 20
if testing.Short() {
perShard = 50000
}
r, err := NewStreamC(10, 3, true, true)
if err != nil {
t.Fatal(err)
}
rand.Seed(0)
input := randomBytes(10, perShard)
data := toBuffers(input)
par := emptyBuffers(3)
err = r.Encode(toReaders(data), toWriters(par))
if err != nil {
t.Fatal(err)
}
// Reset Data
data = toBuffers(input)
all := append(toReaders(data), toReaders(par)...)
ok, err := r.Verify(all)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
err = r.Encode(toReaders(emptyBuffers(1)), toWriters(emptyBuffers(1)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Encode(toReaders(emptyBuffers(10)), toWriters(emptyBuffers(1)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Encode(toReaders(emptyBuffers(10)), toWriters(emptyBuffers(3)))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
badShards := emptyBuffers(10)
badShards[0] = randomBuffer(123)
badShards[1] = randomBuffer(123)
err = r.Encode(toReaders(badShards), toWriters(emptyBuffers(3)))
if err != ErrShardSize {
t.Errorf("expected %v, got %v", ErrShardSize, err)
}
}
func randomBuffer(length int) *bytes.Buffer {
b := make([]byte, length)
fillRandom(b)
return bytes.NewBuffer(b)
}
func randomBytes(n, length int) [][]byte {
bufs := make([][]byte, n)
for j := range bufs {
bufs[j] = make([]byte, length)
fillRandom(bufs[j])
}
return bufs
}
func toBuffers(in [][]byte) []*bytes.Buffer {
out := make([]*bytes.Buffer, len(in))
for i := range in {
out[i] = bytes.NewBuffer(in[i])
}
return out
}
func toReaders(in []*bytes.Buffer) []io.Reader {
out := make([]io.Reader, len(in))
for i := range in {
out[i] = in[i]
}
return out
}
func toWriters(in []*bytes.Buffer) []io.Writer {
out := make([]io.Writer, len(in))
for i := range in {
out[i] = in[i]
}
return out
}
func nilWriters(n int) []io.Writer {
out := make([]io.Writer, n)
for i := range out {
out[i] = nil
}
return out
}
func emptyBuffers(n int) []*bytes.Buffer {
b := make([]*bytes.Buffer, n)
for i := range b {
b[i] = &bytes.Buffer{}
}
return b
}
func toBytes(in []*bytes.Buffer) [][]byte {
b := make([][]byte, len(in))
for i := range in {
b[i] = in[i].Bytes()
}
return b
}
func TestStreamReconstruct(t *testing.T) {
perShard := 10 << 20
if testing.Short() {
perShard = 50000
}
r, err := NewStream(10, 3)
if err != nil {
t.Fatal(err)
}
rand.Seed(0)
shards := randomBytes(10, perShard)
parb := emptyBuffers(3)
err = r.Encode(toReaders(toBuffers(shards)), toWriters(parb))
if err != nil {
t.Fatal(err)
}
parity := toBytes(parb)
all := append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
fill := make([]io.Writer, 13)
// Reconstruct with all shards present, all fill nil
err = r.Reconstruct(all, fill)
if err != nil {
t.Fatal(err)
}
all = append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
// Reconstruct with 10 shards present
all[0] = nil
fill[0] = emptyBuffers(1)[0]
all[7] = nil
fill[7] = emptyBuffers(1)[0]
all[11] = nil
fill[11] = emptyBuffers(1)[0]
err = r.Reconstruct(all, fill)
if err != nil {
t.Fatal(err)
}
shards[0] = fill[0].(*bytes.Buffer).Bytes()
shards[7] = fill[7].(*bytes.Buffer).Bytes()
parity[1] = fill[11].(*bytes.Buffer).Bytes()
all = append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
ok, err := r.Verify(all)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
all = append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
// Reconstruct with 9 shards present (should fail)
all[0] = nil
fill[0] = emptyBuffers(1)[0]
all[4] = nil
fill[4] = emptyBuffers(1)[0]
all[7] = nil
fill[7] = emptyBuffers(1)[0]
all[11] = nil
fill[11] = emptyBuffers(1)[0]
err = r.Reconstruct(all, fill)
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Reconstruct(toReaders(emptyBuffers(3)), toWriters(emptyBuffers(3)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Reconstruct(toReaders(emptyBuffers(13)), toWriters(emptyBuffers(3)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
err = r.Reconstruct(toReaders(emptyBuffers(13)), toWriters(emptyBuffers(13)))
if err != ErrReconstructMismatch {
t.Errorf("expected %v, got %v", ErrReconstructMismatch, err)
}
err = r.Reconstruct(toReaders(emptyBuffers(13)), nilWriters(13))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
}
func TestStreamVerify(t *testing.T) {
perShard := 10 << 20
if testing.Short() {
perShard = 50000
}
r, err := NewStream(10, 4)
if err != nil {
t.Fatal(err)
}
shards := randomBytes(10, perShard)
parb := emptyBuffers(4)
err = r.Encode(toReaders(toBuffers(shards)), toWriters(parb))
if err != nil {
t.Fatal(err)
}
parity := toBytes(parb)
all := append(toReaders(toBuffers(shards)), toReaders(parb)...)
ok, err := r.Verify(all)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Verification failed")
}
// Flip bits in a random byte
parity[0][len(parity[0])-20000] = parity[0][len(parity[0])-20000] ^ 0xff
all = append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
ok, err = r.Verify(all)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("Verification did not fail")
}
// Re-encode
err = r.Encode(toReaders(toBuffers(shards)), toWriters(parb))
if err != nil {
t.Fatal(err)
}
// Fill a data segment with random data
shards[0][len(shards[0])-30000] = shards[0][len(shards[0])-30000] ^ 0xff
all = append(toReaders(toBuffers(shards)), toReaders(parb)...)
ok, err = r.Verify(all)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("Verification did not fail")
}
_, err = r.Verify(toReaders(emptyBuffers(10)))
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
_, err = r.Verify(toReaders(emptyBuffers(14)))
if err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, err)
}
}
func TestStreamOneEncode(t *testing.T) {
codec, err := NewStream(5, 5)
if err != nil {
t.Fatal(err)
}
shards := [][]byte{
{0, 1},
{4, 5},
{2, 3},
{6, 7},
{8, 9},
}
parb := emptyBuffers(5)
codec.Encode(toReaders(toBuffers(shards)), toWriters(parb))
parity := toBytes(parb)
if parity[0][0] != 12 || parity[0][1] != 13 {
t.Fatal("shard 5 mismatch")
}
if parity[1][0] != 10 || parity[1][1] != 11 {
t.Fatal("shard 6 mismatch")
}
if parity[2][0] != 14 || parity[2][1] != 15 {
t.Fatal("shard 7 mismatch")
}
if parity[3][0] != 90 || parity[3][1] != 91 {
t.Fatal("shard 8 mismatch")
}
if parity[4][0] != 94 || parity[4][1] != 95 {
t.Fatal("shard 9 mismatch")
}
all := append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
ok, err := codec.Verify(all)
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("did not verify")
}
shards[3][0]++
all = append(toReaders(toBuffers(shards)), toReaders(toBuffers(parity))...)
ok, err = codec.Verify(all)
if err != nil {
t.Fatal(err)
}
if ok {
t.Fatal("verify did not fail as expected")
}
}
func benchmarkStreamEncode(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := NewStream(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
shards := make([][]byte, dataShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
b.SetBytes(int64(shardSize * dataShards))
b.ResetTimer()
out := make([]io.Writer, parityShards)
for i := range out {
out[i] = ioutil.Discard
}
for i := 0; i < b.N; i++ {
err = r.Encode(toReaders(toBuffers(shards)), out)
if err != nil {
b.Fatal(err)
}
}
}
func BenchmarkStreamEncode10x2x10000(b *testing.B) {
benchmarkStreamEncode(b, 10, 2, 10000)
}
func BenchmarkStreamEncode100x20x10000(b *testing.B) {
benchmarkStreamEncode(b, 100, 20, 10000)
}
func BenchmarkStreamEncode17x3x1M(b *testing.B) {
benchmarkStreamEncode(b, 17, 3, 1024*1024)
}
// Benchmark 10 data shards and 4 parity shards with 16MB each.
func BenchmarkStreamEncode10x4x16M(b *testing.B) {
benchmarkStreamEncode(b, 10, 4, 16*1024*1024)
}
// Benchmark 5 data shards and 2 parity shards with 1MB each.
func BenchmarkStreamEncode5x2x1M(b *testing.B) {
benchmarkStreamEncode(b, 5, 2, 1024*1024)
}
// Benchmark 1 data shards and 2 parity shards with 1MB each.
func BenchmarkStreamEncode10x2x1M(b *testing.B) {
benchmarkStreamEncode(b, 10, 2, 1024*1024)
}
// Benchmark 10 data shards and 4 parity shards with 1MB each.
func BenchmarkStreamEncode10x4x1M(b *testing.B) {
benchmarkStreamEncode(b, 10, 4, 1024*1024)
}
// Benchmark 50 data shards and 20 parity shards with 1MB each.
func BenchmarkStreamEncode50x20x1M(b *testing.B) {
benchmarkStreamEncode(b, 50, 20, 1024*1024)
}
// Benchmark 17 data shards and 3 parity shards with 16MB each.
func BenchmarkStreamEncode17x3x16M(b *testing.B) {
benchmarkStreamEncode(b, 17, 3, 16*1024*1024)
}
func benchmarkStreamVerify(b *testing.B, dataShards, parityShards, shardSize int) {
r, err := NewStream(dataShards, parityShards)
if err != nil {
b.Fatal(err)
}
shards := make([][]byte, parityShards+dataShards)
for s := range shards {
shards[s] = make([]byte, shardSize)
}
rand.Seed(0)
for s := 0; s < dataShards; s++ {
fillRandom(shards[s])
}
err = r.Encode(toReaders(toBuffers(shards[:dataShards])), toWriters(toBuffers(shards[dataShards:])))
if err != nil {
b.Fatal(err)
}
b.SetBytes(int64(shardSize * dataShards))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err = r.Verify(toReaders(toBuffers(shards)))
if err != nil {
b.Fatal(err)
}
}
}
// Benchmark 10 data slices with 2 parity slices holding 10000 bytes each
func BenchmarkStreamVerify10x2x10000(b *testing.B) {
benchmarkStreamVerify(b, 10, 2, 10000)
}
// Benchmark 50 data slices with 5 parity slices holding 100000 bytes each
func BenchmarkStreamVerify50x5x50000(b *testing.B) {
benchmarkStreamVerify(b, 50, 5, 100000)
}
// Benchmark 10 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkStreamVerify10x2x1M(b *testing.B) {
benchmarkStreamVerify(b, 10, 2, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkStreamVerify5x2x1M(b *testing.B) {
benchmarkStreamVerify(b, 5, 2, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 1MB bytes each
func BenchmarkStreamVerify10x4x1M(b *testing.B) {
benchmarkStreamVerify(b, 10, 4, 1024*1024)
}
// Benchmark 5 data slices with 2 parity slices holding 1MB bytes each
func BenchmarkStreamVerify50x20x1M(b *testing.B) {
benchmarkStreamVerify(b, 50, 20, 1024*1024)
}
// Benchmark 10 data slices with 4 parity slices holding 16MB bytes each
func BenchmarkStreamVerify10x4x16M(b *testing.B) {
benchmarkStreamVerify(b, 10, 4, 16*1024*1024)
}
func TestStreamSplitJoin(t *testing.T) {
var data = make([]byte, 250000)
rand.Seed(0)
fillRandom(data)
enc, _ := NewStream(5, 3)
split := emptyBuffers(5)
err := enc.Split(bytes.NewBuffer(data), toWriters(split), int64(len(data)))
if err != nil {
t.Fatal(err)
}
splits := toBytes(split)
expect := len(data) / 5
// Beware, if changing data size
if split[0].Len() != expect {
t.Errorf("unexpected size. expected %d, got %d", expect, split[0].Len())
}
err = enc.Split(bytes.NewBuffer([]byte{}), toWriters(emptyBuffers(3)), 0)
if err != ErrShortData {
t.Errorf("expected %v, got %v", ErrShortData, err)
}
buf := new(bytes.Buffer)
err = enc.Join(buf, toReaders(toBuffers(splits)), int64(len(data)))
if err != nil {
t.Fatal(err)
}
joined := buf.Bytes()
if !bytes.Equal(joined, data) {
t.Fatal("recovered data does match original", joined[:8], data[:8], "... lengths:", len(joined), len(data))
}
err = enc.Join(buf, toReaders(emptyBuffers(2)), 0)
if err != ErrTooFewShards {
t.Errorf("expected %v, got %v", ErrTooFewShards, err)
}
bufs := toReaders(emptyBuffers(5))
bufs[2] = nil
err = enc.Join(buf, bufs, 0)
if se, ok := err.(StreamReadError); ok {
if se.Err != ErrShardNoData {
t.Errorf("expected %v, got %v", ErrShardNoData, se.Err)
}
if se.Stream != 2 {
t.Errorf("Expected error on stream 2, got %d", se.Stream)
}
} else {
t.Errorf("expected error type %T, got %T", StreamReadError{}, err)
}
err = enc.Join(buf, toReaders(toBuffers(splits)), int64(len(data)+1))
if err != ErrShortData {
t.Errorf("expected %v, got %v", ErrShortData, err)
}
}
func TestNewStream(t *testing.T) {
tests := []struct {
data, parity int
err error
}{
{127, 127, nil},
{256, 256, ErrMaxShardNum},
{0, 1, ErrInvShardNum},
{1, 0, ErrInvShardNum},
{257, 1, ErrMaxShardNum},
// overflow causes r.Shards to be negative
{256, int(^uint(0) >> 1), errInvalidRowSize},
}
for _, test := range tests {
_, err := NewStream(test.data, test.parity)
if err != test.err {
t.Errorf("New(%v, %v): expected %v, got %v", test.data, test.parity, test.err, err)
}
}
}