Preheat oven to 375F. Spray a baking pan with non-stick spray.
Combine all the burger ingredients in a large bowl and mix together with your hands or with a fork until all the ingredients are combined. Do not overmix or the burgers will come out tough.
Form the burger patties: Split the venison mixture in half. Then divide each half into 3 equal parts. Form each part into a burger patty that is about the size of your buns.
Preheat your grill pan. Brush the grill pan with the olive oil as you are pre-heating. Place 3 burgers on the pan and cook over medium-high heat for 5 minutes on each side, or until the burgers are nicely browned. If you have a large grill pan, you might be able to fit all 6 burgers in at once.
Place the burgers on the sprayed baking pan and transfer to the oven. Bake for about 10 minutes at 375F for medium burgers, or until the burgers are cooked to your liking. Use a meat thermometer to check the internal temperature, cooking time will vary depending on how thick your burgers are. Remove from the oven and let rest for 5 minutes before eating. Assemble your burger and enjoy!