It certainly is a challenge. Whether or not there's a middle ground generally is a technical question. Again, my background is in law and human rights. All that I can do is look to what the technical community has said.
For years, governments have been saying—and the U.S. government has been prominent in this—that we need to find a way whereby the government can obtain a warrant to access encrypted communication that won't undermine the general security of the information.
Time and time again this has been suggested, and time and time again the people who work on this have said it is not possible. Any illicit means of access, any additional means of access, provides an additional weak point that increases the vulnerability of the system.