I think the Mass State House is pretty good looking.
In the same breath, though, Boston's City Hall is absolutely atrocious, outside and in.
The walls inside some of the hallways are painted this sickening shade of green that would be better suited in my grandmother's farm house than a place of city government. I know it's not a state capitol, but I feel it still deserves mention.