Whisk mayonnaise, horseradish, vinegar, tarragon and tablespoon of dill in a small bowl. Set aside.
In a large wide pot, combine dill, shallot, peppercorns, bay leaf, and 3 tablespoons of salt. Squeeze juice from lemon halves into the pot and add lemon halves themselves. Add 3 quarts water and bring to a boil. Reduce heat to medium and simmer.
Working in batches, drop a few shrimp in at a time until shrimp just begin to curl and turn opaque, about 30 seconds. Transfer to paper towel and let cool.
Chop up shrimp to desired size and toss with horseradish mayonnaise in a medium bowl until evenly coated. Season to taste with salt, pepper, and more horseradish, if desired. Pile salad on toasted potato buns or on top of a green salad!