--- Log opened ma jan 21 00:00:30 2013 00:52 -!- danken1: has quit [Quit: Leaving.] 01:37 -!- markw_ (Mark Wu): has joined #vdsm 02:08 -!- zhshzhou (Zhou Zheng Sheng): has joined #vdsm 02:37 -!- sming (New Now Know How): has joined #vdsm 02:45 -!- shaohef (shaohef): has joined #vdsm 03:19 -!- lvroyce (lvroyce): has joined #vdsm 03:24 -!- osier (osier): has joined #vdsm 03:57 -!- bharata (Bharata B Rao): has joined #vdsm 03:58 -!- sming: has quit [Quit: ChatZilla 0.9.89 [Firefox 10.0.12/20130103094221]] 05:45 -!- deepakcs (Deepak C Shetty): has joined #vdsm 06:15 -!- abaron (Ayal Baron): has joined #vdsm 06:15 -!- abaron_ (Ayal Baron): has joined #vdsm 06:25 -!- lhornyak: has quit [Quit: Leaving.] 06:25 -!- lhornyak (purple): has joined #vdsm 06:52 -!- Humble (Humble Chirammal): has joined #vdsm 07:06 -!- lhornyak: has quit [Quit: Leaving.] 08:03 -!- ykaplan (Yeela Kaplan): has joined #vdsm 08:11 -!- ybronhei: has quit [Quit: Leaving.] 08:12 -!- sming (New Now Know How): has joined #vdsm 08:35 -!- lhornyak (purple): has joined #vdsm 08:48 -!- sming: has quit [Quit: ChatZilla 0.9.89 [Firefox 10.0.12/20130103094221]] 08:49 -!- jlibosva (purple): has joined #vdsm 08:55 -!- odedr (purple): has joined #vdsm 09:10 -!- ybronhei (purple): has joined #vdsm 09:13 -!- lhornyak: has quit [Ping timeout: 252 seconds] 09:26 -!- bharata: has quit [Ping timeout: 252 seconds] 09:29 -!- lhornyak (purple): has joined #vdsm 09:29 -!- apuimedo (Antoni Segura Puimedon): has joined #vdsm 09:32 -!- deepakcs: has quit [Ping timeout: 246 seconds] 09:40 -!- fsimonce (Federico): has joined #vdsm 09:56 -!- abonas (purple): has joined #vdsm 10:00 -!- tjikkun_work (Sander Hoentjen): has joined #vdsm 10:21 -!- bharata (Bharata B Rao): has joined #vdsm 10:22 -!- zhshzhou: has quit [Quit: Leaving.] 10:24 -!- mbetak (purple): has joined #vdsm 10:38 -!- dyasny (Dan Yasny): has joined #vdsm 10:47 < ybronhei> dougsland: ping? 10:56 -!- apuimedo: has quit [Ping timeout: 240 seconds] 10:57 -!- aledvink (Unknown): has joined #vdsm 11:10 -!- mbetak1 (purple): has joined #vdsm 11:11 -!- mbetak: has quit [Ping timeout: 252 seconds] 11:15 -!- apuimedo (Antoni Segura Puimedon): has joined #vdsm 11:17 < dougsland> ybronhei, pong 11:27 -!- mbetak1: has quit [Quit: Leaving.] 11:31 -!- bharata: has quit [Quit: Leaving] 11:37 -!- apuimedo: has quit [Ping timeout: 255 seconds] 11:55 -!- mbetak (purple): has joined #vdsm 12:00 -!- apahim (Amador Pahim): has joined #vdsm 12:02 -!- bala (purple): has joined #vdsm 12:05 -!- granha_bola (Fernando Granha Jernimo): has joined #vdsm 12:08 -!- mbetak: has quit [Ping timeout: 244 seconds] 12:08 -!- lvroyce: has quit [Ping timeout: 256 seconds] 12:13 -!- lvroyce (lvroyce): has joined #vdsm 12:24 -!- mbetak (purple): has joined #vdsm 12:33 -!- bala: has quit [Ping timeout: 248 seconds] 12:33 -!- odedr: has quit [Ping timeout: 256 seconds] 12:39 -!- apuimedo (Antoni Segura Puimedon): has joined #vdsm 12:45 -!- bala (purple): has joined #vdsm 12:53 -!- odedr (purple): has joined #vdsm 12:57 -!- abonas: has quit [Ping timeout: 256 seconds] 13:00 -!- shaohef: has quit [Ping timeout: 252 seconds] 13:04 -!- odedr: has left #vdsm [] 13:04 -!- odedr (purple): has joined #vdsm 13:16 -!- markw_: has quit [Quit: Leaving] 13:18 -!- odedr: has quit [Ping timeout: 248 seconds] 13:28 -!- AlbertoSilva (AlbertoSilva): has joined #vdsm 13:30 -!- mbetak: has quit [Read error: Operation timed out] 13:35 -!- odedr (purple): has joined #vdsm 13:44 -!- granha_bola: has quit [Quit: Leaving] 13:47 -!- mbetak (purple): has joined #vdsm 14:04 -!- saggi: has quit [Read error: Operation timed out] 14:08 -!- ykaplan: has quit [Ping timeout: 246 seconds] 14:20 -!- granha_bola (Fernando Granha Jernimo): has joined #vdsm 14:22 -!- shaohef (shaohef): has joined #vdsm 14:33 -!- ykaplan (Yeela Kaplan): has joined #vdsm 14:39 -!- bala: has quit [Ping timeout: 246 seconds] 14:45 -!- aliguori (Anthony Liguori): has joined #vdsm 14:51 -!- gpadgett (Greg Padgett): has joined #vdsm 14:53 -!- mliu (purple): has joined #vdsm 14:54 -!- lvroyce: has quit [Ping timeout: 248 seconds] 15:12 -!- mliu: has left #vdsm [] 15:18 -!- rharper (Ryan Harper): has joined #vdsm 15:20 -!- apahim: has quit [Quit: Leaving] 15:21 -!- mliu (purple): has joined #vdsm 15:22 -!- mliu: has quit [Client Quit] 15:22 -!- mliu (purple): has joined #vdsm 15:26 -!- mliu: has left #vdsm [] 15:28 -!- apahim (Amador Pahim): has joined #vdsm 15:33 -!- saggi (purple): has joined #vdsm 15:36 < saggi> danken: ping, back from the brink of despair. What did I miss? 16:04 -!- lhornyak: has quit [Ping timeout: 248 seconds] 16:05 -!- lhornyak (purple): has joined #vdsm 16:07 -!- acathrow (Aic): has joined #vdsm 16:13 -!- odedr: has quit [Ping timeout: 244 seconds] 16:14 -!- odedr (purple): has joined #vdsm 16:15 -!- odedr: has quit [Client Quit] 16:17 <@danken> saggi: why despair? you have a progressive president being sworn in, we, on the contrary.... 16:18 < saggi> danken: Was down most of last week. Caught a really annoying flu. 16:19 <@danken> ouch. I thought you were just doing your thing, ignoring emails. ;-) 16:19 <@danken> saggi: master is broken on el6 (unit tests failing) - I did not check why, something about json 16:20 -!- odedr (purple): has joined #vdsm 16:20 < saggi> danken: "Somthing about Json" is the name of comrom I'm writing. 16:20 < saggi> romcom 16:22 -!- apuimedo: has quit [Ping timeout: 248 seconds] 16:25 -!- abonas (purple): has joined #vdsm 16:27 <@danken> could you take a look at the broken test? hopefully we'd soon have ut running on el6 too 16:27 <@danken> saggi: feel well, I need to go, my wife is not feeling well herself. 16:27 < saggi> danken: I will, take care 16:29 -!- danken: has quit [Quit: Leaving.] 16:35 -!- abaron_: has quit [Quit: Leaving] 16:41 -!- shaohef: has quit [Ping timeout: 252 seconds] 17:16 -!- odedr: has quit [Quit: Leaving.] 17:17 -!- lhornyak: has quit [Quit: Leaving.] 17:20 -!- osier: has quit [Read error: Connection reset by peer] 17:32 -!- abaron: has quit [Ping timeout: 248 seconds] 17:37 -!- ykaplan: has quit [Remote host closed the connection] 17:39 -!- mbetak: has quit [Ping timeout: 244 seconds] 17:40 -!- jlibosva: has quit [Quit: Leaving.] 17:43 < saggi> fsimonce: ping 17:43 < fsimonce> saggi, pong 17:43 < saggi> fsimonce: http://gerrit.ovirt.org/#/c/9749/ 17:43 < saggi> lets turn that -1 upside down 17:47 < fsimonce> I still don't see the point of doing it on all the HSMs 17:49 < fsimonce> I understand that it might be negligible for performance but still HSMs are not supposed to be writing the metadata anyway 17:55 -!- abonas: has quit [Ping timeout: 248 seconds] 17:59 -!- aledvink: has quit [Ping timeout: 248 seconds] 18:01 < saggi> fsimonce: Doesn't matter 18:01 < saggi> any hsm can become spm 18:02 < saggi> one day 18:02 < saggi> if it only dreams 18:02 < saggi> and does it's homework 18:03 < saggi> fsimonce: premature optimization is the root of all evil. 18:05 < saggi> fsimonce: That's Knuth, not me. If he was wrong, you would have gotten a 256c check. 18:05 < saggi> 2.56$ 18:08 < fsimonce> it's not optimization, it's correctness. and for example I'd define optimization to "do the homeworks" on the HSMs. 18:08 < saggi> What? What I am doing to the correct thing 18:09 < saggi> It's unsupported be design 18:09 < saggi> we don't want to block ourselves 18:09 < saggi> if we ever need a mailbox on file domains 18:09 < saggi> were dead 18:09 < fsimonce> I added a suggestion on gerrit 18:09 < saggi> Also, even HSMs read the MD using directIO 18:09 < saggi> for obvious reasons 18:10 < fsimonce> you can check it on creation and startSpm, that's where we need it 18:10 < saggi> No, because MD reading is done using directio 18:10 < saggi> fsimonce: ^^ 18:10 < saggi> even on HSM 18:11 < fsimonce> then the test is wrong, you should check if you're able to read using directio 18:12 < saggi> fsimonce: if the test fails it fails on open(2) 18:12 < saggi> fsimonce: The reason I write and not read is becase direcreadlines() fails if the file doesn't exist 18:12 < fsimonce> either you need to test it or not, if you do, you need the proper test (open with directio), or you don't and that's it 18:13 < saggi> fsimonce: So if I change it so I write a blank string 18:13 < saggi> will this please you? 18:13 < fsimonce> I also don't like the fact that this something else that can potentially make testing harder (if you instantiate a domain object it needs to write to the disk) 18:14 < saggi> Hmmm 18:14 < fsimonce> well I'd be pleased if the spm check for directio for writing and if the hsms check for directio for reading 18:15 < saggi> fsimonce: OK, Host A has samba 3.2.1 Host B has samba 3.2.10 18:15 < saggi> Now lets say that on version 3.2.8 samba added support of direct IO 18:15 < fsimonce> then on one of them startSpm would fail 18:15 < saggi> if you create the domain on B you have a problem with host A 18:15 < saggi> because A can't read the MD 18:16 < saggi> So it will fail with a weird error 18:16 < saggi> No, as to changing it to just open 18:16 < saggi> *Now 18:16 < saggi> you have to use open(O_CREATE | O_DIRECT) 18:16 < saggi> which means the file will be created anyway 18:17 < saggi> which is write 18:17 < saggi> open will fail 18:17 < saggi> but the file will be created 18:17 < saggi> So, by changing it to Just open 18:17 < saggi> You add another oop method 18:18 < saggi> with negligible benefits 18:19 < saggi> fsimonce: ^^^ 18:20 < saggi> fsimonce: Anyway, thinking about it. I need to remove the unlink() as it can create a race. The file can just stay there IMO. 18:20 < fsimonce> yes 18:20 < saggi> fsimonce: Any other complaints? 18:21 < fsimonce> (yes was about the race, not about the rest) 18:21 -!- mbetak (purple): has joined #vdsm 18:21 < fsimonce> write the open(O_CREATE | O_DIRECT) thingy on gerrit 18:22 < saggi> fsimonce: I can just change it now 18:22 < saggi> as I am sending another version anyway 18:22 < saggi> I will do it 18:22 < saggi> I just think it's redundant 18:22 -!- mbetak: has quit [Client Quit] 18:22 < fsimonce> I'm thinking on how not add an additional oop 18:23 -!- ybronhei: has quit [Ping timeout: 252 seconds] 18:25 < saggi> fsimonce: I can have directWriteLines() use truncate() instead of write() if data is "" 18:26 < fsimonce> I'm searching if there's any way to detect if the call failed for the O_DIRECT part 18:26 < saggi> There is 18:26 < fsimonce> anyway it's something that probably belongs in DirectFile 18:26 < saggi> the error is EINVAL 18:26 < fsimonce> ah right 18:26 < saggi> Your problem is that you don't want to do a write 18:26 < saggi> Which is fine 18:27 < fsimonce> yes, you can do a read 18:27 < fsimonce> and catch that 18:27 < saggi> read will fail if the file didn't already exist 18:27 < saggi> with ENOENT 18:27 < saggi> it is checked before direct IO support 18:27 < fsimonce> ok fine, but the metadata file should be there 18:27 < fsimonce> (otherwise it's not a domain) 18:29 < saggi> You do realize that this is utterly redundant and binds the check with the current structure of the domain. 18:29 < saggi> but 18:29 < saggi> fuck it 18:29 < saggi> I'm not in storage anymore 18:29 < saggi> you can do whatever you want 18:31 < saggi> Lets do more work and make things more complex to save a 1 block write every few minutes. VDSM is so efficient, I wouldn't want to bug it down with 1 redundant write. 18:31 < saggi> bogg 18:35 < saggi> fsimonce: ping 18:35 < fsimonce> :) 18:35 < saggi> fsimonce: For create(), what do you want to do. I want to stop it in _preCreateValidation(). But we don't have an md file yet 18:35 < saggi> and I don't want to create one 18:35 < saggi> if it fails validation 18:36 < saggi> can I, in that case, use a random file? 18:36 < fsimonce> sure 18:36 < saggi> fsimonce: I'm not going to delete it 18:36 < saggi> fsimonce: races and shit 18:37 < saggi> fsimonce: We don't have the lock yet 18:37 < saggi> (we can't get it without directio) 18:38 < fsimonce> it's not about doing things more complex, it's about not breaking assumptions for any small matter that we're working on. just because it takes less time. 18:39 < fsimonce> races? it's the spm, what race are you thinking of? 18:40 < fsimonce> create domain race? 18:40 < fsimonce> (between DCs on the same share) 18:41 < fsimonce> it's up to you, you can use a fixed file name or a random one (which in this case I'd suggest to remove) 18:41 < saggi> fsimonce: I'll use a fixed name 18:42 < saggi> fsimonce: And assumptions are the mother of all f**kups 18:42 < saggi> fsimonce: Assumptions is what made the engine design their task system without support of non SPM tasks, 18:45 < fsimonce> yeah "assumption" is vague, it's not the assumption that you think it is, "assumption" as "axiom" 18:46 < fsimonce> I'm not assuming that we'll never write the metadata from the HSMs, I'm saying that your patch is not a valid reason to break the "axiom" of HSMs not writing the metadata 18:47 < fsimonce> maybe I'm underestimating the importance of your patch though 18:47 < saggi> fsimonce: This is why I didn't write to the metadata 18:47 < saggi> fsimonce: I would never open the md file with "w" on an HSM 18:49 < fsimonce> still you're testing O_DIRECT for writing when you need to test if for reading, we need to "assume" that it's the same 18:50 < saggi> fsimonce: As I explained, I test it for opening 18:50 < fsimonce> if you "assume" that it's the same, just read also from the spm, and that's it 18:50 < saggi> the reason I used write 18:50 < saggi> is because I used a file that might not exist 18:50 < saggi> because I didn't want to use anything important 18:52 -!- #rhev,#vdsm acathrow: has quit [Ping timeout: 248 seconds] 18:53 < saggi> EINVAL is returned from open(2) not from read(2) or write(2). directReadLines() opens with O_DIRECT | O_RDONLY this means that if the file doesn't exist the error would have been ENOENT. 18:53 < saggi> This means I have to use O_DIRECT | O_CREATE. directWriteLines() uses O_DIRECT | O_CREATE | O_RDWR . Which is close enough. 18:54 < fsimonce> if you think it's easier to convince abaron, ask him, I just feel really hard to give a +1 18:55 < saggi> fsimonce: The problem is, if you check if the file exists before I choose whether to read or to write, I do another oop operation, which is currently more expensive then just writing. 18:55 < saggi> since oop spaces are limited as is 18:55 < saggi> now, because you were so stressed that VDSM might just not be able to run VMs with this extra write 18:56 < saggi> I've changed it now so you can set a parameter to the method whether it reads or writes. 18:56 < saggi> so the create() flow and the ctor flows are covered. 18:57 < saggi> This is what one might call premature optimization. 18:57 < saggi> that one being me 18:57 < fsimonce> I explained that I'm not stressed about the extra writes 18:58 < saggi> I'm not going to argue with abaron. I've completely lost faith in hes abilities to make sane decisions. 18:58 < fsimonce> I'm stressed about you not explaining why you want to test something that you won't use :) ...why testing RW if then you just use RO, and in doing so you break the axioms 18:59 < fsimonce> :) 18:59 < saggi> fsimonce: As I said, I can ad an oop method that just opens with O_CREATE | O_DIRECT 18:59 < saggi> fsimonce: Just looks a bit redundant. 18:59 < saggi> I'll call it directTouch 19:00 < fsimonce> is everything going through DirectFile? 19:00 < saggi> Yes 19:00 < saggi> I'll just do With DirectFile(path, "dw"): pass 19:01 < saggi> which will open and clsoe 19:01 < saggi> close 19:01 < saggi> under oop of course 19:01 < fsimonce> why don't you catch there the EINVAL and raise the exception (maybe a specific one DirectIOError) 19:02 < saggi> Because that is what EINVAL means 19:02 < saggi> I just need to translate it in the domain level to a domain error 19:02 < saggi> because in the future 19:02 < saggi> more things might raise this error 19:02 < fsimonce> that's why you need to translate it as soon as possible into DirectIOError 19:03 < saggi> I don't understand why I need to translate it? open(2) didn't find it appropriate to use something else 19:03 < saggi> who am I to change that? 19:04 < saggi> If someone tries to open 19:04 < fsimonce> ah sorry I thought you said that you were worried that EINVAL was going to be used by other things 19:04 < saggi> No 19:04 < saggi> I meant StorageDomainTargetUnsupported() 19:04 < fsimonce> ok, how do you filter that by a read that raises an EINVAL? 19:04 < saggi> because it's not read 19:04 < saggi> it's own 19:04 < saggi> open() 19:05 < fsimonce> yes so then you need to change directReadLines() 19:05 < fsimonce> because f.readlines can raise EINVAL too 19:05 < saggi> If directReadLines() returns EINVAL because of read() it's a bug in DirectFile() 19:05 < saggi> As I said, I'll create a new oop method that is more concise 19:06 < saggi> I think this will make us both happy 19:06 < saggi> that just opens __direct_io_test__ with O_DIRECT | O_CREATE but doesn't read() or write() 19:09 < saggi> fsimonce: OK? 19:10 < saggi> fsimonce: http://fpaste.org/0Jnw/ ? 19:11 < fsimonce> I'm not overly happy, I think that we can get this all for free (no need to create/test/validate anything). We can catch the issue in DirectFile (both for ro and rw) and bubble the specific issue up. We'll get for free that all the flow will fail reporting the correct error and we don't add/change/create anything. 19:11 < saggi> It will fails somewhere 19:12 < saggi> you don't know 19:12 < saggi> and this can change 19:12 -!- abaron (Ayal Baron): has joined #vdsm 19:12 < fsimonce> vdsm should be able to cope with that 19:12 < fsimonce> directRead/WriteLines can fail at any time anyway 19:12 < saggi> fsimonce: Look, I am not, repeat, *NOT*, going to add a storage_exception to any of my classes. This entire file is a layering violation. 19:13 < fsimonce> that's not what I suggested 19:13 < saggi> There is no reason for DirectFile to have any use exported exceptions 19:13 < saggi> user 19:13 < saggi> it's and internal implementation detail if there ever was one 19:13 < fsimonce> I said you should bubble it up... if you write a DirectFile class, you can add its own exception 19:13 < saggi> and...? 19:14 < saggi> How does this solve anything? 19:14 < fsimonce> it's not very much internal if it's in the name :) *Direct*File 19:14 < saggi> you should use open_ex() to abstract what file object you get 19:14 < fsimonce> you bubble a DirectIOError which is caught in the domain class 19:15 < saggi> No 19:15 < saggi> I'm not going to add another exception because you don't want to add another more specific method. 19:15 < saggi> There is being lazy and there is being wrong. 19:16 < fsimonce> and you happen to be both this time :) 19:16 < saggi> directReadLines() exists only because you can't use instances through oop 19:16 < saggi> which is logical 19:16 < saggi> but when oop goes away 19:16 < saggi> we are left with a weird Exception 19:16 < fsimonce> ok, I'm fine with the directTouch 19:17 < saggi> think about it now try DirectFile() needs to expect OSError, IOError and DirectIOError? 19:17 < saggi> Just because of this? 19:18 < saggi> fsimonce: By the way, specific exceptions are going away. 19:18 < saggi> fsimonce: What's good enough for the standard library is good enough for me. 19:18 < saggi> fsimonce: And it's impossible to support 19:19 < fsimonce> well said 19:19 < saggi> We will have some 19:21 < saggi> But having StoragePoolCreationError() and StorageDomainCreationError() is utter stupidity. First of all you never create them both in the same method so you don't need the first part. and just having CreationError() is stupid because you know you called create so using them is as effective as throwing Exception() 19:23 < saggi> fsimonce: Anyway, I'm going to see if there is still food in the cafeteria. I'll send a revised patch after I eat. Take care, and thanks! 19:27 -!- odedr (purple): has joined #vdsm 19:29 -!- mbetak (purple): has joined #vdsm 19:31 -!- mbetak: has left #vdsm [] 19:42 < fsimonce> saggi, yup I ran for food too 20:05 -!- lhornyak (purple): has joined #vdsm 20:18 -!- acathrow (Aic): has joined #vdsm 20:23 -!- mode/#vdsm: by ChanServ 20:23 -!- danken (purple): has joined #vdsm 20:28 -!- rharper: has quit [Quit: Ex-Chat] 20:28 -!- rharper (Ryan Harper): has joined #vdsm 20:45 -!- acathrow is now known as acathrow_mtg 21:20 < saggi> fsimonce: ping 21:33 -!- danken: has quit [Quit: Leaving.] 22:01 -!- acathrow (Aic): has joined #vdsm 22:01 -!- acathrow (Aic): has joined #vdsm 22:11 -!- acathrow (Aic): has joined #vdsm 22:43 -!- odedr: has quit [Quit: Leaving.] 23:21 -!- acathrow (Aic): has joined #vdsm --- Log closed di jan 22 00:00:30 2013