The district would be subservient to the state for taxes, lease contracts etc. but it's entirely possible to treat the individual buildings as "sovereign territory of the District of Columbia" just like how the little townhouse in London is Ecuadorian territory, and thousands of other embassies around the world like it.

Once you walked off the sidewalk, or off the third floor elevators, or through that door, etc, you would be on "federal turf" so to speak. It's not about a geographical circle.