The shmoo plot is just the name for measuring something (such as perf) over a range of parameter space. The simplest and most straightforward application is to pick a parameter or two that you don’t know what value they should be using, do the shmoo over the range of parameter space, and then set the knobs at whatever values give you the optimal measurement.

Usually though, you have to iterate. Doing shmoos along the way can help with understanding the effects of code changes, help understand how the hardware works, and it can sometimes help identify what code changes you might need to make. A simple abstract example might be I know what my theoretical peak bandwidth is, but my program only gets 30% of peak. I suspect it has to do with how many registers are used, and I have a knob to control it, so I turn the knob and plot all possible register settings, and find out that I can get 45% of peak with a different value. Now I know it was partially registers I was limited by, but I also know to look for something else too. Then I profile, examine the code, maybe refactor or adjust some things, hypothesize, test, and then shmoo again on a different knob or two if I suspect something else is the bottleneck.

Bayesian optimization is very good an optimizing black box Knob in shmoo plot usually called meta parameter. Assuming each probing is costly they allow you to find the optional combination of value for many knob (dimension) with a minimum of probing