Yes, they have been arrested and there have been instances in the last few years. Mainly, some agents have gone after high-profile defectors. For example, Hwang Jang-yop, who was essentially the number two within North Korea, defected, and there were instances where agents were found pursuing him. The average North Korean defector is often not the target of these agents. It's these high-profile defectors who end up getting caught. The national intelligence service of South Korea actively pursues them and monitors them. Especially for the North Koreans who are applying in Thailand, for example, who are seeking to come to South Korea, there's a very intensive interview process during which they cross-check and triangulate against a database of essentially 26,000 other defector stories, and against other intelligence they have within North Korea.
On October 2nd, 2014. See this statement in context.