(CNN) -- The Justice Department will never prosecute journalists for doing their jobs, and recent probes into national security leaks targeted government officials, not reporters, Attorney General Eric Holder said in opening remarks to a Senate committee Thursday.

Holder, amid a cloud of controversy for investigations in recent years involving The Associated Press and Fox News, said he has launched a review of existing Justice Department guidelines on investigations involving press, and he has met with reporters to discuss those guidelines. He said the conversation is not static.

"The department goal in investigating leak cases is to identify and prosecute government officials who jeopardize government secrets," Holder told the Senate Appropriations Committee during a wide-ranging budget hearing that included questions about the federal prison system and the closing of Guantanamo Bay.

He added that as long as he is at the Justice helm, he will never prosecute a reporter for doing her or his job.

Sen. Mark Kirk also raised questions about a Thursday report that the National Security Agency and FBI were monitoring Americans' phone records. He asked specifically whether Holder could assure him that no member of Congress had been monitored, as it might give the executive branch leverage of the legislative branch.

Holder responded that it wasn't an appropriate venue to answer the question, to which Kirk said the appropriate answer was, "No, we stayed in our lane, and I assure you we did not spy on members of Congress."

Sen. Barbara Mikulski, chairwoman of the committee, interrupted the back-and-forth to say that the matter deserved a briefing before the entire Senate, and involving the NSA and Holder.

Sen. Richard Shelby, R-Alabama, the ranking GOP member of the committee, opened his remarks by saying the Justice Department was "mired in a controversy of late" that raised questions about the Justice Department's "adherence to the rule of law" and Holder's ability to lead. He further said Americans deserved an attorney general "not distracted by controversies of his own making."

Holder emphasized he was "fully engaged" the efforts to resolve these problems and evaluates his own performance on a daily basis.

"I have not done a perfect job. I think I've done a good job, but I think I could do better," he said, adding that his recent meetings with journalists were aimed at formulating new policies and regulations "and hopefully get that behind us."

Responding to Shelby's query about whether there would be a tipping point, at which Holder might need to step down, Holder -- who has suggested he might not serve for President Barack Obama's entire second term -- said he had more goals to accomplish before he sat down with Obama to discuss a transition.

"The tipping point might be fatigue," Holder told Shelby. "You get to a point where you just get tired."

Sen. Dianne Feinstein did not continue the line of questioning regarding the leaks but opened her statement by defending Holder and lamenting that the hearing was used to berate him.

"I believe in your integrity," she said. I believe you're a good attorney general. I believe you've had undue problems that are hard to anticipate. I believe you're responding as best you possibly could."

Holder is under fire for two instances, in particular. The first involves his Justice Department obtaining two month of phone records from The Associated Press as part of an investigation into the news agency's May 2012 coverage of a foiled airline bomb plot in Yemen. The second case involves Justice obtaining the phone records, e-mails and security badge information of Fox News' James Rosen, who reported on classified intelligence about North Korea in 2009.

No reporters were singled out as potential criminals in the AP case, but in the Fox case, an FBI agent said Rosen might be an "aider and abettor and/or co-conspirator" to disclosing secret information.

The Rosen case has been of most interest to Holder's critics because of a May 15 remark he made to Congress about the leaks.

"With regard to the potential prosecution of the press for the disclosure of material, that is not something that I've ever been involved in or heard of or would think would be a wise policy," Holder said.

Republican Rep. Darrell Issa called Holder's statement "a lie, by most people's standards," and the GOP-controlled House Judiciary Committee is investigating whether the attorney general lied under oath.

The White House and Justice Department have both issued statements saying Rosen was never prosecuted, so any assertion that Holder lied is wrong.

The Justice Department has also said that Holder recused himself from the AP probe because he had been interviewed about the leak during the investigation, but Republicans say the statement was missing a key piece of information: When did he recuse himself?

After hearing concerns that the Justice Department's investigations had put reporters at a legal risk for simply doing their jobs, Holder sat down with various news executives last week.

"We expressed our concerns that reporters felt some fear for doing their jobs, that they were concerned about using their e-mail, using their office telephone and that we need to have the freedom to do their job," The Washington Post Executive Editor Martin Baron said after the meeting.

Holder told NBC News on Wednesday that he would not step down amid criticism over security leaks investigations.