{"id":747,"date":"2018-03-30T06:25:06","date_gmt":"2018-03-30T12:25:06","guid":{"rendered":"http:\/\/www.gergltd.com\/home\/?p=747"},"modified":"2018-04-12T02:13:03","modified_gmt":"2018-04-12T08:13:03","slug":"changing-dropout-on-the-fly-during-training-time-test-time-in-keras","status":"publish","type":"post","link":"https:\/\/www.gergltd.com\/home\/changing-dropout-on-the-fly-during-training-time-test-time-in-keras\/","title":{"rendered":"Changing dropout on the fly (during training time, test time) in keras"},"content":{"rendered":"

I was doing an experiment that involved Monte-Carlo (MC) dropout. I had to change the rate of dropout and have it dropout during test time.<\/p>\n

I was able to turn dropout on during\u00a0test time by making a new function and passing in the learning_phase tensor.<\/p>\n

# This code allows dropout to run during test time.\r\nnewLayer = K.function([model.layers[0].input, K.learning_phase()], [model.output])\r\nlayer_output = newLayer([x, 1])[0] # Dropout on<\/pre>\n
To change dropout from train time to test time I did the following:<\/p>\n
# This code allows you to change the dropout\r\n# Load model from .json\r\nmodel.load_weights(filenameToModelWeights) # Load weights\r\nmodel.layers[-2].rate = 0.04  # layer[-2] is my dropout layer, rate is dropout attribute\r\nmodel = keras.models.clone(model) # If I do not clone, the new rate is never used. Weights are re-init now.\r\nmodel.load_weights(filenameToModelWeights) # Load weights\r\nmodel.predict(x)\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
I was doing an experiment that involved Monte-Carlo (MC) dropout. I had to change the rate of dropout and have it dropout during test time. I was able to turn dropout on during\u00a0test time by making a new function and passing in the learning_phase tensor. # This code allows dropout to run during test time. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-747","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/posts\/747","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/comments?post=747"}],"version-history":[{"count":2,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/posts\/747\/revisions"}],"predecessor-version":[{"id":749,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/posts\/747\/revisions\/749"}],"wp:attachment":[{"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/media?parent=747"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/categories?post=747"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gergltd.com\/home\/wp-json\/wp\/v2\/tags?post=747"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}