Place all the ingredients for the pimento cheese in a large bowl and stir well to combine.
Meanwhile, boil the potatoes until fork-tender. Drain.
Add 1 cup of the prepared pimento cheese, butter, warmed milk, and black pepper to the boiled potatoes.
Use a potato masher to mash to your desired consistency (adding more warm milk if desired).
Serve warm or place the potatoes in a baking dish, cover, and refrigerate.
Refrigerate remaining pimento cheese and use it as a spread for crackers, sandwiches, or vegetables.