weird "index out of range" in strconv.formatBits

metronome

unread,

Aug 16, 2023, 10:31:18 PM8/16/23

to golang-nuts

Hi,

We ran into a weird out of range issue of strconv.formatBits, hope someone can shed a light on what could be the root cause, any comment is highly appreciated.

problem description:

random out of range at code, most of the time the indexing is a huge int but we observed at least one exception (#2).

#1: runtime error: index out of range [18446744073709449339] with length 200

      #2: runtime error: index out of range [102511] with length 200

Wild guesses:

1. The machine code seems to suggest it's unlikely a data race or memory corruption? But perhaps 

relevant registers, like R10, had been saved and restored, then it might be due to stack corruption?

Given that R12 is scratch reg, is it possible that R12 is clobbered somehow, say, by signal handling?

===================================================================

   0x0000000000495b0a<+810>:        mov    %rdi,%r10

   0x0000000000495b0d<+813>:        shr    %rdi
   0x0000000000495b10<+816>:        mov    %rax,%rsi
   0x0000000000495b13<+819>:        movabs $0xa3d70a3d70a3d70b,%rax
   0x0000000000495b1d<+829>:        mov    %rdx,%r11
   0x0000000000495b20<+832>:        mul    %rdi
   0x0000000000495b23<+835>:        shr    $0x5,%rdx
   0x0000000000495b27<+839>:        imul   $0x64,%rdx,%r12
   0x0000000000495b2b<+843>:        sub    %r12,%r10
   0x0000000000495b2e<+846>:        lea    (%r10,%r10,1),%rax
   0x0000000000495b32<+850>:        lea    0x1(%rax),%rax
   0x0000000000495b36<+854>:        nopw   0x0(%rax,%rax,1)
   0x0000000000495b3f<+863>:        nop
   0x0000000000495b40<+864>:        cmp    $0xc8,%rax
   0x0000000000495b46<+870>:        jae    0x495c8f <strconv.formatBits+1199>

Dan Kortschak

unread,

Aug 16, 2023, 11:04:35 PM8/16/23

to [email protected]

Go version?
Function invocation context?

Kurtis Rader

unread,

Aug 16, 2023, 11:56:17 PM8/16/23

to metronome, golang-nuts

Insufficient information. Version of Go? Since formatBits is private we need to see the actual code of a call to a public API that resulted in the call to formatBits that failed. Also, show us the literal panic stack. Showing us the assembly code with no context is not useful. If your program using pure Go (statically linked) or Cgo? Wild guesses, what I used to call SWAGS (silly wild ass guesses) as a Unix support engineer, are seldom useful. If your guess is scientifically informed that is a different matter but you should be able to articulate why you think your guess is more likely to be true than a random coin flip.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ff804d4c-24ee-480d-8ed1-219f9b8d7cbcn%40googlegroups.com.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

metronome

unread,

Aug 17, 2023, 1:43:13 AM8/17/23

to golang-nuts

Thanks for commenting, a few supplements.

# 1. Version of Go?

We observed the issue with both go1.20.5 and go1.18.10 on linux/amd64 (centos)

# 2. Context?

All panics we observed so far are from either

strconv.FormatInt -> strconv.formatBits chain, or

strconv.FormatUint -> strconv.formatBits chain

where the base is always 10.

// typical call site, toId is an "*int64".

if com_count > 1 {

com_string = anchor + "," + strconv.FormatInt(*toId, 10)

}

# 3. If your program using pure Go (statically linked) or Cgo?

Binary was built with CGO_ENABLED=1 and -buildmode=exe.

All panic call sites we observed so far are "pure go", that is, no C calling go path.

# 4. panic stack trace

panic: runtime error: index out of range [18446744073708732603] with length 200

goroutine 1 [running]:
strconv.formatBits({0x0?, 0x0?, 0x0?}, 0xc09e00b750?, 0x1?, 0x1?, 0x0?)
/usr/lib/go-1.20/src/strconv/itoa.go:140 +0x4b9
strconv.FormatInt(0x0?, 0xc07393df80?)
/usr/lib/go-1.20/src/strconv/itoa.go:29 +0xa5

...

Dan Kortschak

unread,

Aug 17, 2023, 1:46:16 AM8/17/23

to [email protected]

Have you eliminated the possibility of races? Have you built with
CGO_ENABLED=0?

metronome

unread,

Aug 18, 2023, 1:32:18 AM8/18/23

to golang-nuts

Thanks.

>> Have you eliminated the possibility of races?

We're trying to extract a smaller case for reproducing (perhaps impractical, as the frequency of the panic is way low) then apply the race detector, enabling race check online is not an option now, sadly.

>> Have you built with CGO_ENABLED=0?

Building with CGO_ENABLED=0 succeeded, does that mean the binary's runtime behavior has nothing to do with CGO, deploying

a CGO_ENABLED=0 binary online is not an option as well, for now (We are trying, but not sure if we can make it happen).

Dan Kortschak

unread,

Aug 18, 2023, 2:18:10 AM8/18/23

to [email protected]

On Thu, 2023-08-17 at 23:32 -0700, metronome wrote:
> > > Have you built with CGO_ENABLED=0?
> Building with CGO_ENABLED=0 succeeded, does that mean the binary's
> runtime behavior has nothing to do with CGO, deploying
> a CGO_ENABLED=0 binary online is not an option as well, for now (We
> are trying, but not sure if we can make it happen).

Do you get the same behaviour with CGO_ENABLED=0?

metronome

unread,

Sep 1, 2023, 10:48:10 AM9/1/23

to golang-nuts

We waited for two weeks, but the panic never resurfaced, will provide further updates if it reoccurs or as soon as we have more information.

Thanks Dan and Kurtis for looking into it.

世宇杨

unread,

Apr 29, 2025, 5:43:43 PM (6 days ago) Apr 29

to golang-nuts

We have also encountered the same problem. May I ask if you have found out the cause?

Reply all

Reply to author

Forward