cmd/compile/internal/pgo: error parsing profile (for pgo) after scaling #73640
Comments
gabyhelp
added
the
BugReport
cherrymui
changed the title
|
Hmmm, apparently the github.com/google/pprof/profile package doesn't preserve the start_line field? It probably should, as it is in the proto definition. cc @prattmic |
cherrymui
added
NeedsFix
|
I think this is just a bad error message. I suspect what actually happened is that your profile still has samples, but every sample has value. For example, here is what I see when running your program on a profile I collected from a simple spinning program. Before: After: Two things to notice here.
What happens in PGO parsing:
We should probably bail out if (3) creates an empty graph, the same way we bail out if the entire profile is empty. Though it might be nice to warn users, or even fall back to using @matthinrichsen-wf can you confirm that I think the problem with this scaling approach is that the I think this approach would work if you stripped out the |
|
@prattmic Yep! All the samples/count are 0. I don't think I quite understand how to strip out |
|
I don't think there is any nice helper function. I think it would work fine to simply remove the |
|
Gotcha! Like this? package main
import (
"bytes"
"os"
"time"
"github.com/google/pprof/profile"
)
func removeI[T any](in []T, index int) []T {
if index < 0 || index >= len(in) {
return in
}
out := make([]T, 0, len(in)-1)
out = append(out, in[:index]...)
out = append(out, in[index+1:]...)
return out
}
func main() {
b, err := os.ReadFile(`profile.pb`)
if err != nil {
panic(err)
}
p, err := profile.ParseData(b)
if err != nil {
panic(err)
}
sampleTypes := make([]*profile.ValueType, 0, len(p.SampleType))
for i, st := range p.SampleType {
if st.Type == `cpu` && st.Unit == `nanoseconds` {
for _, s := range p.Sample {
s.Value = removeI(s.Value, i)
}
continue
}
sampleTypes = append(sampleTypes, st)
}
p.SampleType = sampleTypes
// scale the profile to reduce the size
p.Scale(1 / (float64(time.Millisecond)))
p.Scale(float64(time.Millisecond))
buf := &bytes.Buffer{}
err = p.Write(buf)
if err != nil {
panic(err)
}
os.WriteFile(`scaled.pb`, buf.Bytes(), 0644)
}I've used it in |
|
I now notice that when I use |
|
I also notice that I'm curious if that's to be expected and if it will impact the ability of pgo to optimize well? |
|
That profile has no samples. With no samples PGO will completely ignore it and do nothing. I suspect you just scaled away every sample (were there any samples with more than 1000 count?). Try a less extreme scaling. |
|
@prattmic Looking at the original, no there we no counts above 1,000. I did find there's a scaling method that seems to work: package main
import (
"bytes"
"os"
"time"
"github.com/google/pprof/profile"
)
func scaleCPUSamples(p *profile.Profile, ratio float64) {
ratios := make([]float64, len(p.SampleType))
for i, st := range p.SampleType {
if st.Type == `cpu` && st.Unit == `nanoseconds` {
ratios[i] = ratio
} else {
ratios[i] = 1
}
}
p.ScaleN(ratios)
}
func main() {
b, err := os.ReadFile(`profile.pb`)
if err != nil {
panic(err)
}
p, err := profile.ParseData(b)
if err != nil {
panic(err)
}
// scale the profile to reduce the size
scaleCPUSamples(p, 1/(float64(time.Millisecond)))
scaleCPUSamples(p, float64(time.Millisecond))
buf := &bytes.Buffer{}
err = p.Write(buf)
if err != nil {
panic(err)
}
os.WriteFile(`scaled.pb`, buf.Bytes(), 0644)
}^ this reduces profile size and can still be used by pgo and seems to solve the problem. For context: the original profile is >100MB. |
Go version
go version go1.24.2 darwin/arm64
Output of
go envin your module/workspace:What did you do?
I have a profile which I'd like to use for PGO, however it is exceptionally large. I scaled the profile to reduce the size. However after scaling, the profile cannot be used for PGO. I've included an example scaler:
I then attempted to use PGO with scaled profile
go build -pgo scaled.pb main.goWhat did you see happen?
Received the following error:
The original profile can be used with pgo, as expected:
go build -pgo profile.pb main.goWhat did you expect to see?
I expected both the scaled and non-scaled profiles to be usable for PGO.
The text was updated successfully, but these errors were encountered: