Gunmen entered the office following the blast and members of Crisis Response Unit arrived at the site to fight them.
The siege of the building ended as security forces killed all five assailiants, the Interior Ministry said in a statement.
The blast happened close to the office of Counterpart International development organization in Shahr-i-Naw area, the Interior Ministry said.
At least five people including four civilians and one police were killed in the attack. 24 more were wounded, the Interior Ministry said in a statement.
A total of 200 employees of the organization were rescued, the statement said.
Taliban claimed responsibility for the attack, saying the target was the Counterpart office, which is funded by the US.
US Ambassador Bass condemned the attack, saying the targeted organization helps local communities, trains journalists and supports the Afghan people.
The United Nations Assistance Mission also condemned the assault.
“Today’s attack particularly deplorable, hitting civilians helping Afghans & taking place during Ramadan,” UNAMA said on Twitter.